KHE diary for 2024
==================

At the end of 2023 I was working on bringing KHE back into good
form for high school timetabling.  Specifically I was just
finishing off a rewrite of khe_sm_correlation.c.

1 January 2024.  Finished off the reimplementation of khe_sm_correlation.c
  (documentation and code).  It's all good and ready to test.

  I'm currently looking into the case of ordinary demand tixels
  being attached even though there are no constraints demanding
  their tasks be assigned.  It's clear from the documentation
  (e.g. of KheSolnMinMatchingWeight) that I've hitherto been
  expecting that all event resources will have an accompanying
  assign resource constraint.  I need something more nuanced.

2 January 2024.  Spent the day thinking about the issue documented
  yesterday.  The simple way forward is to only create ordinary
  demand monitors for tasks that have an assign resources constraint.
  This could be the "originating monitor" for these demand monitors.

  Started writing a section of the Matchings chapter entitled
  "Separate matching and integrated matching" which seems to be
  making the best of a not great situation.  

3 January 2024.  Thinking about how the matching code is divided
  between the platform and the solvers.

  Started documenting the new matching plan.  I've finished the
  platform part of it.

4 January 2024.  Finished documenting the solver side of things.
  There are just the two functions, KheMatchingBegin and
  KheMatchingEnd, but there is a lot of explanation.  It all
  needs a careful audit, then it will be time to implement it.

5 January 2024.  I've finished updating the documentation; I've
  moved every affected bit to its new home.  And I've done a lot
  of implementing, whose effect is to move everything interesting
  into khe_sm_matching.c.  I have a clean compile, but khe_sm_matching.c
  needs an audit.

6 January 2024.  Audited everything in khe_sm_matching.c and
  implemented the few things that were left over.  All in good
  shape but another audit would be good.

7 January 2024.  Did some auditing, more is needed.

8 January 2024.  Finished auditing khe_sm_matching.c, started testing.
  KheAvoidClashesAugment was not implemented.  I've returned to an
  old implementation but it has a problem with excluding days.  I still
  have to work out what to do about that.  Also I had to bring back
  clashing times in resource timetable monitors, which was fine
  except now I'm keeping the clashing times sorted, so that they
  can be relied on when repairing.  I did a quick review of the
  documentation of the resource timetable monitor, and found a few
  things to fix, including getting rid of some withdrawn functions.

  KheAvoidClashesAugment had a problem with excluding days, so I
  fell back on a set of repairs, each of which simply unassigned
  one of the clashing tasks.  Not deep but may be good enough.

9 January 2024.  Done some tests, got this:

    [ "AU-BG-98", 12 threads, 12 solves, 12 distinct costs, 172.9 secs:
      1.00725 4.00936 6.00958 7.00742 7.00834 7.00997 7.01037 9.00795
      10.00763 11.00988 12.00700 12.00889
    ] best soln (cost 1.00725) has diversifier 11

  The best is quite good, but overall there could be more consistency.
  Here's a full run (10 mins per instance):

  Instance     KHE14x8     KHE24x12    Best (from Gerhard's web site)
  -------------------------------------------------------------------
  AU-BG-98     4.00524      1.00795    0.00128 (GOAL)
  AU-SA-96     6.00006      1.00023    0.00000 (GOAL)
  AU-TE-99     2.00140      1.00192    0.00020 (GOAL - optimal)
  BR-SA-00     1.00051      0.00048    0.00005 (Sorensen et al - optimal)
  BR-SM-00    22.00129     12.00131    0.00051 (Sorensen - optimal)
  BR-SN-00     4.00243      0.00204    0.00035 (Sorensen - optimal)
  DK-FG-12     0.02046      0.01759    0.01263 (GOAL)
  DK-HG-12           -     12.02932   12.02330 (GOAL)
  DK-VG-09    12.03257      2.02692    2.02323 (GOAL)
  UK-SP-06     changed     29.01046    2.01410 (Dudek)
  FI-PB-98     1.00024      1.00007    0.00000 (Kyngas and Nurmi)
  FI-WP-06     0.00041      0.00021    0.00000 (GOAL)
  FI-MP-06     0.00125      0.00114    0.00077 (GOAL)
  GR-H1-97     0.00000      0.00000    0.00000 (Pimmer)
  GR-P3-10     0.00006      0.00002    0.00000 (Gogos and Valouxis)
  GR-PA-08     0.00021      0.00013    0.00003 (GOAL - optimal)
  IT-I4-96     0.00197      0.00076    0.00027 (GOAL - optimal)
  KS-PR-11     0.00116      0.00147    0.00000 (Demirovic and Musliu)
  NL-KP-03     0.03919      0.02950    0.00199 (GOAL)
  NL-KP-05           -      3.01457    0.00425 (GOAL)
  NL-KP-09           -      6.11190    0.01620 (GOAL)
  ZA-LW-09    16.00000     16.00010    0.00000 (Gogos et al.)
  ZA-WD-09     6.00000     17.00000    0.00000 (Sorensen et al.)
  ES-SS-08     0.01287      0.00967    0.00335 (Sorensen - v. optimal)
  US-WS-09    untested      0.00520    0.00101 (Klemsa - optimal)
  --------------------------------------------------------------------

  Overall run time was 31.0 minutes, which over 25 instances is
  74.4 seconds per instance on average.  This is quite comparable
  with the run I did on 27 December 2023, where I found 24 solutions
  to each instance and ran for 44.5 minutes altogether.  In fact,
  here is that longer run again:

  Instance     KHE14x8     KHE24x24    Best (from Gerhard's web site)
  -------------------------------------------------------------------
  AU-BG-98     4.00524      1.00795    0.00128 (GOAL)
  AU-SA-96     6.00006      0.00018    0.00000 (GOAL)
  AU-TE-99     2.00140      1.00192    0.00020 (GOAL - optimal)
  BR-SA-00     1.00051      0.00048    0.00005 (Sorensen et al - optimal)
  BR-SM-00    22.00129      9.00155    0.00051 (Sorensen - optimal)
  BR-SN-00     4.00243      0.00156    0.00035 (Sorensen - optimal)
  DK-FG-12     0.02046      0.01759    0.01263 (GOAL)
  DK-HG-12           -     12.03001   12.02330 (GOAL)
  DK-VG-09    12.03257      2.02664    2.02323 (GOAL)
  UK-SP-06     changed     29.01046    2.01410 (Dudek)
  FI-PB-98     1.00024      0.00034    0.00000 (Kyngas and Nurmi)
  FI-WP-06     0.00041      0.00016    0.00000 (GOAL)
  FI-MP-06     0.00125      0.00094    0.00077 (GOAL)
  GR-H1-97     0.00000      0.00000    0.00000 (Pimmer)
  GR-P3-10     0.00006      0.00000    0.00000 (Gogos and Valouxis)
  GR-PA-08     0.00021      0.00011    0.00003 (GOAL - optimal)
  IT-I4-96     0.00197      0.00063    0.00027 (GOAL - optimal)
  KS-PR-11     0.00116      0.00147    0.00000 (Demirovic and Musliu)
  NL-KP-03     0.03919      0.02889    0.00199 (GOAL)
  NL-KP-05           -      2.01466    0.00425 (GOAL)
  NL-KP-09           -      6.11190    0.01620 (GOAL)
  ZA-LW-09    16.00000     16.00010    0.00000 (Gogos et al.)
  ZA-WD-09     6.00000     17.00000    0.00000 (Sorensen et al.)
  ES-SS-08     0.01287      0.00875    0.00335 (Sorensen - v. optimal)
  US-WS-09    untested      0.00520    0.00101 (Klemsa - optimal)
  --------------------------------------------------------------------

  Overall run time was 61.4 minutes.  There are pluses and minuses
  here from the comparable run on 27 December 2023, but overall the
  results are either much the same or slightly better.

  Previously I worked out what all the operations that changed
  monitors were:

    KheMonitorSetBack
    KheMonitorDetachFromSoln
    KheMonitorAttachToSoln

    KheClusterBusyTimesMonitorSetCutoffIndex
    KheClusterBusyTimesMonitorSetCutoffTime (non-atomic)
    KheClusterBusyTimesMonitorSetNotBusyState
    KheClusterBusyTimesMonitorClearNotBusyState
    KheClusterBusyTimesMonitorSetMultiplier
    KheClusterBusyTimesMonitorSetMinimum
    KheClusterBusyTimesMonitorResetMinimum

    KheLimitBusyTimesMonitorSetCeiling

    KheLimitWorkloadMonitorSetCeiling

    KheLimitActiveIntervalsMonitorSetCutoffIndex
    KheLimitActiveIntervalsMonitorSetCutoffTime
    KheLimitActiveIntervalsMonitorSetNotBusyState
    KheLimitActiveIntervalsMonitorClearNotBusyState

    KheEventTimetableMonitorMake (omit? does not affect cost)

    KheGroupMonitorMake
    KheGroupMonitorDelete
    KheGroupMonitorAddChildMonitor
    KheGroupMonitorDeleteChildMonitor

  I was thinking that these should be recorded in paths so that they can
  be undone, but I'm not sure now and I'm not going to rush into it.

  Started work on two-level indexed solution sets (called Hard and
  Soft).  Done the boilerplate for the new types, now I have to
  update Soft and insert a Hard above each use of Soft.  The indexing
  is now done in variables of type int rather than type KHE_COST.

10 January 2024.  Finished the two-level indexed solution sets.  Then
  went to work on a KheMarkEnd matching bug.  It's a hard one to track
  down.  Got some interesting debug output showing that a rogue ordinary
  workload demand monitor has appeared:

    [ A0 06910 <ordinary_demand_monitor> 1.00000 3Wed:Day.10+0:Caretaker ]

  The point is that the solve was nowhere near being up to 3Wed, so
  this is looking like a complete ring-in.

11 January 2024.  Still working on the KheMarkEnd matching bug.
  Actually it's not such a ring-in, because there are workload
  demand monitors for those kinds of times e.g. "1Mon2:1Tue1+40"
  which means 10 days after 1Mon1, ec.

  The last act on monitor 06910 before the crash is
 
    Domain {CT_23}:   [ A0 06910 <ordinary_demand_monitor> 1.00000 3Wed:Day.10+0:Caretaker ]

  This is quite strange, why would we limit the domain to a single
  resource when we are about to undo?  Or is the problem that we
  have not undone before we test the matching?  Perhaps the matching
  needs to be undone last?

12 January 2024.  Still working on the KheMarkEnd matching bug.
  The matching is already being undone last.  Finally I have some
  debug output that seems to show clearly what has gone wrong:

    [ KheMatchingMarkBegin 46890 with lower bounds 545
      [ A0 06910 0.00000 0x7f0ce4fcc338 3Wed:Day.10+0: Caretaker : {0-1, 4-24} ]
    ] KheMatchingMarkEnd 47566 with lower bounds 545
      [ A0 06910 1.00000 0x7f0ce4fcc338 3Wed:Day.10+0: Caretaker : {23} ]

  So when KheMatchingMarkBegin is called, monitor 06910's task has
  domain Caretaker and the monitor itself has {0-1, 4-24} for its
  domain (presumably all caretakers).  But at the end, when we should
  be back where we were, monitor 06910's task has domain Caretaker but
  the monitor itself has domain {23}.  This could be because the
  task is assigned to something; I have to look into that now.

  Actually the case has just got murkier:

    [ KheMatchingMarkBegin 46890 with lower bounds 545
      [ A0 06910 0.00000 0x7f108dd6a338 3Wed:Day.10+0: Caretaker : {0-1, 4-24} ]
	task 3Wed:Day.10  -> 3Tue:Day.13  -> /CT_23/
    ] KheMatchingMarkEnd 47566 with lower bounds 545
      [ A0 06910 1.00000 0x7f108dd6a338 3Wed:Day.10+0: Caretaker : {23} ]
	task 3Wed:Day.10  -> 3Tue:Day.13  -> /CT_23/

  So now it seems that 3Wed:Day.10 has always been assigned, indirectly,
  to CT_23; but initially its domain does not reflect that assignment.
  So what about an invariant check every time either changes?

14 January 2024.  Fixed the 546 vs 545 bug.  The problem was that I
  was writing

      if( HaArrayCount(task->all_monitors) > 0 )

  where what I really meant was

      if( KheSolnHasMatching(task->soln) )

  This didn't matter previously, because every task got some monitors
  when the matching was installed; but now only tasks with hard assign
  resource monitors get monitors, and so there can be a matching even
  when HaArrayCount(task->all_monitors) is 0.

  Finally we're back to solving nurse rostering instances:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 10 dist costs, 299.3 secs:
      0.01845 0.01890 0.01900 0.01900 0.01940 0.01950 0.01970 0.01970
      0.01990 0.01995 0.02010 0.02095
    ] best soln (cost 0.01845) has diversifier 8

  Not a great result but at least we are running.

  I've gone back to the nurse rostering paper and run that.  It was
  slow but there were no crashes.  I've run the whole thing with -n2
  or whatever.

15 January 2024.  Finished reading through the paper.  What's needed
  now are more results, and good results.  I'm starting by looking at
  COI-Azaiez.  KHE20x8 got cost 0 in 0.8 seconds, which is a lot
  better than KHE24 is getting at the moment.

  I've done a whynot run.  Something seems to be very odd about
  the swaps it is trying.  But the main problem is that there
  don't seem to be any Move operations at all, and the right
  fix throughout here is to move unassigned tasks to these
  underloaded resources.

16 January 2024.  Sorted out the problem with COI-Azaiez.  It was
  blocking swaps that it should have allowed, and only trying tasks
  that needed assignment (so that if the resource needed assignment
  it was not happening):

    [ "COI-Azaiez", 12 threads, 12 solves, 6 distinct costs, 22.0 secs:
      0.00000 0.00003 0.00004 0.00004 0.00004 0.00005 0.00005 0.00005
      0.00005 0.00006 0.00010 0.00010
    ] best soln (cost 0.00000) has diversifier 4

  This is just what we were looking for.

  Found a problem with dynamic when running COI-Ikegami-2.1:

    KheInstanceTime: i (-1344826576) out of range (0 - 89)

  This ultimately goes back to cluster constraint Constraint:6,
  which has these attributes:

    <TimeGroups>
      <TimeGroup Reference="TG:Empty" Polarity="negative"/>
      <TimeGroup Reference="1Sat1-2"/>
    </TimeGroups>
    <Minimum>0</Minimum>
    <Maximum>1</Maximum>

  A literal reading of the spec suggests that the first of these
  time groups is always active.  It's hard to say what this is about,
  but it may be an attempt to encode some history.  Anyway I've added
  the following to the specification of XESTT cluster busy times
  constraints (and so implicitly to limit active intervals constraints):

    "It is probably better if each time group is non-empty.  However, the
     rules just given are valid for empty time groups; they specify that
     a positive empty time group is always inactive, and a negative empty
     time group is always active."

  And I've modified the dynamic code so that it now accepts empty time
  groups.  Let's hope it evaluates them in accordance with the spec.

  Working on COI-Millar-2.1, where I'm getting best cost 200 where
  previously I was getting 0.  I can see why ejection chains are not
  repairing the last two defects (tight constraints).  This is a good
  candidate for dynamic, let's see what's happening there.

  Dynamic was not running because resources had workload demand
  monitors and dynamic thought it could not handle them.  I've
  changed it to ignore those monitors, and guess what - I've got
  COI-Millar-2.1 down to 0.00000.  Great stuff.

17 January 2024.  Working on the COI-ORTEC01 bug today.

    INT_SUM_COST(ARC:NA=s1000:1/1Thu:L/NA=s1000:1, NO_MAX, 2)

  The 2 is closed state.  Evidently it's wrong, because this node
  has only one child, but why?  This seems to be the reason:

    KheDrsExprIntSumCostCheckClosed: actual closed state (0) != stored value (1)
    [ INT_SUM_COST(ARC:NA=s1000:1/1Thu:E/NA=s1000:1, NO_MAX, 1) pi 13561 oc 0
      [ ASSIGNED_TASK(1Thu:E.0 := -, -, val 0) p 0x7fcfa8ddc4b8 pi 13560 oc 0 ]
    ]

  Before any expressions are actually opened by KheDrsSolveOpen, this
  INT_SUM_COST expression has closed value 1, even though its only child
  (which is closed at this point) has value 0.

  I'm not sure, but the problem may be the call to KheDrsExprLeafClear(e)
  in KheDrsExprOpen.  It can change the value of the ASSIGNED_TASK node
  from 1 to 0 without informing the parents at all.  But then, when it
  does this it is already open, so the parent does not care what its
  value is.

  The latest debug output seems to be suggesting that I'm doing updates
  to expressions that are marked closed.  But now that I have an open
  flag in each expression, I can easily check for that.  Another
  question:  was the relevant task opened, was it meant to be opened?
  It is in the right day range, but if it is assigned some resource
  that is not being reassigned, then it should not be opened.

18 January 2024.  Still working on the COI-ORTEC01 bug.  I got it
  down to a single thread.  The task currently causing trouble is
  1Fri:E.0, which is part of a larger task:

      1Wed:E.0{1Thu:E.0{}, 1Fri:E.0{}}

  This larger task does not unassign because it lies partly
  outside the interval of interest.  So then it is not opened,
  because only unassigned tasks are opened.  So this explains
  why it is closed during solving.  But then why are we trying
  to do things to it during search, even though it's closed?

  I can see that dtd->closed_drd and drd->closed_dtd will
  both be non-NULL, because KheDrsTaskUnAssign will not be
  called.  So KheDrsResourceOnDayIsFixed will return true
  because drd has a closed assignment.

  The basic questions are (1) is it OK for the expressions to
  be closed during solving?  It would be hard to open them,
  given that the task is not unassigned.  (2) Assuming that
  the expressions are closed during solving, how do we make
  the solve code handle them correctly?  Is it as simple as
  not updating closed expressions?

  Should we keep BUSY_TIME and similar expressions closed in
  these cases?  Arguably it would be more efficient since they
  would not be updated during solving and their closed cost
  would always be there.  The rule would be

     drd->closed_dtd != NULL ==> drd->external_today not opened
     dtd->closed_drd != NULL ==> dtd->external_today not opened

  But efficiency hardly matters here, since it is all so
  peripheral.  What matters is correctness.

  Even if a task is unassigned it can remain closed because
  it is grouped with tasks that are out of range.  We can't
  assign such a task, but we won't be assigning it because
  its enclosing mtask and shift will not be opened.

19 January 2024.  Thinking about the COI-ORTEC01 bug.  It is quite
  right that the task should not open, but the question then is,
  why does it get treated as though it is open during the solve?
  And the answer seems to be that its resource is fixed to it and
  the resource constraints at any rate are opened then.

  Perhaps the principled thing is to not open anything that will
  not change, and to exclude it from the search space.  We could
  still included it in solutions, but we'd be recognizing that
  it is closed and needs no expression updating.

  Each drd can be open or closed independently of its enclosing dr,
  in fact dr can be defined to be open when it contains at least
  one open drd.  But each dtd is open or closed depending entirely
  on whether its enclosing dt is open or closed.

  If a drd is closed on an open day, it means that the assignment
  of its resource is fixed on that day.  We can include that
  assignment in solutions but the expressions will already have
  factored it in.

  A preassigned task that is initially assigned can be treated
  as closed, since it will not change.

  How the bug originated:

    Resource dr was assigned to task dt, and dt is partly inside
    and partly outside the open day range [da .. db].  So dt is
    not open on da, but dr is open on that day and it has a closed
    assignment to a dtd of dt.  When that closed assignment is made,
    dtd's expressions get updated, even though they are closed.

  The best way seems to be to include closed assignments in
  solutions but to not open their expressions and not update them.

  Did "cp khe_sr_dynamic_resource.c save_khe_sr_dynamic_resource.c".

  Added !KheTaskIsPreassigned(dt->task, &r) to KheDrsResourceOnDayOpen,
  so that now preassigned tasks that are assigned a resource are not
  unassigned, which means that they will remain closed.  Also added
  if( drd->closed_dtd == NULL ), so drd's expressions are opened only
  if drd is open (is unassigned at the end of KheDrsResourceOnDayOpen).

  The main remaining problem is which code to execute, and which not
  to execute, during expansion?  We should not execute code that
  sets and clears leaf expressions that are not open.  For example,
  KheDrsResourceSignatureMake should still be called but there it
  calls KheDrsResourceOnDayLeafSet and KheDrsResourceOnDayLeafClear,
  and those calls should be skipped when the assignment is closed.

20 January 2024.  I've decided that the presence of a closed_drd
  and closed_dtd should be tested for and should prevent updating
  any relevant expressions - they will also be closed.  I've
  implemented that and I've run COI-ORTEC01 without crashing.

21 January 2024.  Overnight run of COI finished this morning without
  crashing, but with many "inconsistent monitor" warnings.  Fixing
  those is the next job.  Funny it never leads to a crash.

21 January 2024.  I worked this out yesterday:

  These two lines for monitor ARC:NA=s1000:1/4Fri:N/NA=s1000:1 look odd:

    6#       os search: 0.00000 + 0.01000 - 0.00000
    6#       os search: 0.01000 + 0.01000 - 0.00000

  While searching, we seem to be reporting a cost twice.  We need to
  find out where these have originated.  Here is more detail on these
  two lines:

    6# 4Fri:52 os search: 0.00000 + 0.01000 - 0.00000
    6#  4Fri os search: 0.01000 + 0.01000 - 0.00000

  The first is a shift signer.  I need to check, but very likely it
  should not contribute directly to the monitor cost.  I need to
  work out who does contribute and when.

  Today I corrected this problem by guarding the update rerun cost
  code with "if( dsg == NULL || dsg->debug_eval_if_rerun )".  This
  was there before but some idiot took it away.

  I've compared KHE24x24's COI results with KHE18's COI results from
  my previous paper.  KHE24x24 is out of sight better, but it does
  not yet reach the "success in practice" threshold.

  Looking into COI-SINTEF.  It's small instance, the main problems
  are on weekends where the nurses to assign are basically fixed.
  COI-SINTEF has limit workload constraints.  It would be a great
  test when adding these to dynamic.  That would very likely get
  the cost down to zero.  Each workload is either 750 or 975,
  after division by the gcd (75) this is 10 and 13.  But since
  we are not using tries we perhaps don't need the gcd.

22 January 2024.  Off-site backup today (but not new version).

  Took another look at how KHE is running on high school instances.
  No crashes, and got the same results as those reported on 9 January
  2024.  As reported there, a few of the instances are coming out
  poorly (e.g. UK-SP-06) and need looking into.

23 January 2024.  Started work on limit workload monitors.  I've
  reviewed KHE_DRS_EXPR_FLOAT_SUM, KHE_DRS_EXPR_FLOAT_DEV, and
  KHE_DRS_EXPR_WORK_TIME; they all seem fine.  I've also reviewed
  the expression tree in the doc, and it seems to accurately reflect
  the semantics.  So the next step is to put all the pieces together.

24 January 2024.  Brought back the code for building expression trees
  for limit workload monitors.  All good except now I have to
  think about dominance testing.

25 January 2024.  My evaluation of the int sum cost domination test
  functions:

    * KheDrsExprNotifySigners is good as is.  It calls
      KheDrsExprIntSumCostNotifyCoverSigners and
      KheDrsExprNotifyResourceSigners.

    * KheDrsExprIntSumCostNotifyCoverSigners is irrelevant.

    * KheDrsExprNotifyResourceSigners is relevant and calls
      KheDrsExprNeedsDayEval to get the dom test it needs.

    * KheDrsExprNeedsDayEval seems to be OK, except that it calls
      KheDrsExprDayDomTest.

    * KheDrsExprDayDomTest has a large switch that needs looking into.

    * KheDrsExprIntSumCostChildDomTest is correct for limit workload monitors.

    * KheDomTestTypeActual may need adapting.

    * KheDrsExprIntSumCostDomTest has an unassigned_child_count parameter
      which needs adapting; it is subtracted from eisc->max_limit, so for
      limit workload monitors it must be a workload.  Also this function
      accesses tables so that is not going to work.  We probably need a
      completely different function at this point.

    * KheDrsExprIntSumCostDayDomTest also has unassigned_child_count.

    * KheDrsIntSumCostNeedsEval also has unassigned_child_count, although
      its boolean result is probably correct now.

    * KheDrsIntSumCostNeedsShiftEval and KheDrsIntSumCostNeedsShiftPairEval
      are only called for int sum cost expressions derived from event
      resource monitors, so not relevant.

26 January 2024.  I think that there might be a problem with limit
  busy times expression trees.  Their root node is an INT_SUM_COST
  expression, but that assumes that each child has value 0 or 1.
  Whereas, with limit busy times monitors each child's value can
  be an arbitrary non-negative integer.

  I've decided to introduce types KHE_LIMIT_EFFORT_CONSTRAINT and
  KHE_LIMIT_EFFORT_MONITOR and use them to implement limit busy
  times and limit workload constraints and monitors.  There will
  be a boolean flag in each constraint saying which type it is,
  so no functionality will be lost, but very often no-one will
  care which type it is, so there will be a big saving in code.
  In particular, I'll save code when I add history and cutoffs.

  Finished making limit effort constraints the union of the other two,
  and limit effor monitors the union of the other two.

27 January 2024.  Decided not to proceed with limit effort constraints
  and monitors.  It will be less disruptive to add cutoffs to limit
  workload monitors, then copy them to limit busy times monitors.
  And history is not really needed at the moment anyway.  So I've
  commented out everything I did yesterday related to limit effort
  constraints and monitors.

28 January 2024.  Working carefully through khe_sr_time_sweep.c,
  bringing it up to date with the new approach to time sweep.
  Have clean compile but still lots to do.

  Added cutoff time functions to event resource monitors.  The
  interfaces, implementations, and documentation are all done.
  The implementation sometimes uses attaching and detaching,
  which is not great, but we're making it do.

29 January 2024.  Working on cutoff times and time sweep.  I've
  audited and slightly changed the implementations of cutoff times
  in the event resource monitors, and expanded the documentation.
  I've also changed "cutoff time" to "sweep time" throughout.

  I discovered that several resource monitors already had functions
  for finding first and last times.  So I've withdrawn those functions
  in favour of the new ones.  All done including the documentation.

30 January 2024.  Working on cutoff times and time sweep.  I've
  finished reviewing khe_cluster_busy_times_monitor.c and
  khe_limit_active_intervals_monitor.c.  By not changing the
  definition of cutoff index (now sweep index) I seem to have
  been able to use the previous code without change, except for
  a change to the meaning of a NULL sweep time which is very
  easy to implement.

  Added a KHE_BUSY_AND_IDLE type which passes busy count, idle count,
  workload, and open count.

31 January 2024.  Working on sweep times.  Finished limit busy times
  monitors, and audited them and monitored time groups, all good.

1 February 2024.  Implemented the revised plan for time ranges and
  sweep times, which is to have only KheMonitorTimeRange,
  KheMonitorSetSweepTime, and KheMonitorSweepTimeRange defined
  in khe_platform.h, with the subclass versions of them defined
  in khe_interns.h.  All done, just documentation needed now.

2 February 2024.  Simplified cluster by merging the three sweep
  fields into just two, sweep_time and sweep_index.  Did a careful
  audit of the whole thing.  Seems good.  Then did the same for
  limit active intervals monitors; the code is basically the same.

  Updated the sweep time documentation.

3 February 2024.  Finished off the changes to limit workload monitors.
  This included a new function, KheResourceTypeMaxWorkloadPerTime,
  which helps to adjust workloads when there is a sweep time.  All
  done and documented.

  Worked through khe_sr_time_sweep.c, to audit it and bring it up to
  date with the new approach to time sweep.  All done, ready to test.

4 February 2024.  Changed limit busy times and limit workload monitors
  so that the defective time groups appear in a particular fixed order.
  This helps with traversing the list and trying repairs.  All done and
  documented.

  Testing COI_SINTEF now.  This is what I was getting before:

    [ "COI-SINTEF", 1 solution, in 0.6 secs: cost 0.00006 ]

  Fixed a bug (just a cut and paste error, in the end) in limit workload 
  monitors.   Now I'm getting this:

    [ "COI-SINTEF", 1 solution, in 0.6 secs: cost 0.00001 ]

    [ "COI-SINTEF", 12 threads, 12 solves, 7 distinct costs, 6.2 secs:
      0.00001 0.00003 0.00004 0.00004 0.00005 0.00007 0.00007 0.00007
      0.00008 0.00010 0.00010 0.00010
    ]

  So there is already an improvement.  That's good.  I'm doing a
  full COI run, or at least the 5 minute limit part of it.  A few
  results are marginally worse.  This one is quite a lot better:

    [ "COI-ERMGH", 12 threads, 24 solves, 24 distinct costs, 272.4 secs:
      0.01423 0.01552 0.01575 0.01603 0.01606 0.01648 0.01666 0.01671
      0.01698 0.01701 0.01737 0.01740 0.01819 0.01853 0.01863 0.01869
      0.01921 0.01999 0.02026 0.02099 0.02188 0.02211 0.02611 0.02860
    ]

  The previous 5 minute run result had cost 2171.  But this one got worse:

    [ "COI-Valouxis-1", 12 threads, 24 solves, 11 distinct costs, 201.2 secs:
      0.00120 0.00140 0.00140 0.00140 0.00140 0.00140 0.00140 0.00160
      0.00200 0.00200 0.00200 0.00220 0.00220 0.00220 0.00220 0.00220
      0.00240 0.00260 0.00260 0.00280 0.00300 0.00300 0.00320 0.00380
    ]

  The previous five-minute solution had cost only 40.

    [ "COI-MER", 12 threads, 24 solves, 24 distinct costs, 7.1 mins:
      0.51552 0.54046 0.60648 0.61503 0.61645 0.62243 0.64206 0.67559
      0.69731 0.72807 0.73472 0.76168 0.76932 0.80952 0.81639 0.82162
      0.83697 0.84447 0.84604 0.84649 0.85116 0.89836 0.99999 0.99999
    ]

  Much worse than 38855, the previous 5-minute solution.

  Now abandoning the vlsn search immediately if cost is 0.  COI-Azaiez
  should benefit.

  Did a complete COI run.  There are winners and losers, on the
  whole I would guess things are slightly better.

5 February 2024.  Off-site backup today.

  Did a quick initial swing through the INT_SUM_COST code in dynamic.
  There are parts that treat the children as having integer values,
  but other parts seem to assume that the values are all 0 or 1.  So
  there is a lot of reviewing to do to see whether INT_SUM_COST can
  have general integer values.

  Also looked over the documentation.  At some point I seem to
  have decided that a counter monitor has children whose values
  are either 0 or 1, and a workload monitor has children whose
  values can be any non-negative integer.  So perhaps we need

      Old name                      New name
      -------------------------------------------------------
      KHE_DRS_EXPR_INT_SUM_COST	    KHE_DRS_EXPR_COUNT
      KHE_DRS_EXPR_INT_SEQ_COST     KHE_DRS_EXPR_SEQUENCE
      -                             KHE_DRS_EXPR_TOTAL
      -------------------------------------------------------

  with the third type handling both limit busy times monitors
  and limit workload monitors (being the root node of both).
  It can have its own dominance test, which can be very simple,
  just tradeoff dominance, no tables.  There is no minimum
  or maximum limit, the unweighted cost is just the sum of
  the children's values.

  Need types INT_DEV_SUM and FLOAT_DEV_SUM which are the
  child nodes in limit busy times and limit workload
  expression trees respectively.  In each case the root
  is KHE_DRS_EXPR_TOTAL.

  If we just start off from scratch implementing these
  new node types, we'll be able to bypass a lot of old
  code whose meaning can be difficult to fathom.  There
  seems to be no need for correlated dominance tests in
  these types, for example.

  We'll need a max_workload attribute in WORK_TIME
  and WORK_DAY, which can be summed over all open
  nodes to influence the deviation when there is
  a lower limit.  And we'll need to keep track of
  this sum as chldren are opened and closed.

6 February 2024.  I've been wondering whether there is any point
  in trying to bring dynamic closer to the main implementation,
  for example by having a BUSY_AND_IDLE type and merging all
  the single time expressions into one, whose value is a busy
  and idle value.  Another left-field idea is to distribute
  the expression tree dynamic code over the existing monitors.

  Added max_workload_per_time field to drs.  It's initialized, and
  used when initializing objects of type KHE_DRS_EXPR_WORK_TIME.

  KheDrsOpenChildrenAtOrAfter shows that there are really three
  kinds of children:

    * Children that remain closed for the duration of one solve.  Their
      values are summarized as eisc->history + eisc->eisc_closed_state.

    * Children that remain open for the duration of one solve.
      These are all history_after children.  Their values are
      summarized as eisc->history_after.

    * Children that are open when the solve begins but that
      gradually become closed as it progresses, in the sense
      of having a definite state.  But they are called open.

  It would be good to have clearer names for these three cases
  (or four considering that the last case has two parts), e.g.

      FULL_CLOSED
      SEMI_CLOSED
      SEMI_OPEN
      FULL_OPEN

  But SEMI_CLOSED vs SEMI_OPEN depends on where the solve is up to.
  Lower bounds have to be adjusted by the sum of the maximum possible
  values of the semi-open children.  This is done for cluster by
  KheDrsOpenChildrenAtOrAfter, since the maximum value in that
  case is 1 per semi-open child.  But for limit busy times and limit
  workload monitors the maximum value can be larger than 1,
  in fact, a lot larger.  KheDrsOpenChildrenAtOrAfter is also
  called several times when constructing dominance tests.  So
  there is another roadblock.

  Another way out would be to not report underloads when there
  is at least one open child.  This would handle limit workload
  constraints quite well, since they are largely about overloads.
  Suppose we do this.  Then we just need to account for the fact
  that there are child values that are neither zero nor one.

7 February 2024.  I've been thinking about the best way to divide
  the work among the different expression types.  What about this:

    COST       Sum the chilren's values and apply the cost function.
               The assumption here is that each child provides a
	       value which is a deviation.

    INT_DEV    Sum the children's integer values and calculate
               a deviation, which is the value of the expression.

    FLOAT_DEV  Sum the children's float values and calculate an
               integer deviation, which is the value of the expression

  The problem is that it doesn't really gel with INT_SEQ_COST, and
  we really do need INT_SEQ_COST.

  Another, different idea is to generatize KheDrsOpenChildrenAtOrAfter
  so that it returns a value that we might call

    KheDrsExprOpenChildrenMaxValue

  which is an upper bound on the contribution that open childen could
  make to the determinant.  Also KheDrsIntSumCostUpdateLU assumes
  that child values are either 0 or 1.  In fact the whole ld/ud
  thing assumes this.  But does it need to?

  KheDrsIntSumCostAdjustLimits also assumes that each child has
  value 0 or 1.

  Had an interesting idea, to make the value of each expression
  be a range (either int or float).  It may be a wide range
  when the expression is open, but it must be a single point
  when the expression is closed.  This single point is not
  necessarily what is stored in the signature.  Rather, the
  signature holds some internal state from which the value
  interval can be determined.

    Node type           Open value           Closed value
    ----------------------------------------------------------------
    BUSY_TIME           [0, 1]               0 or 1
    FREE_TIME           [0, 1]               0 or 1
    WORK_TIME           [0, M]               0 <= val <= M
    OR                  [0, 1]               0 or 1
    AND                 [0, 1]               0 or 1
    SUM                 [sigma cx, sigma cy] sigma cx <= val <= sigma cy
    ----------------------------------------------------------------

  We don't necessarily aim for the tightest possible open value.
  We just want a reasonable range that bounds all possible closed
  values.  As the children of SUM nodes close, we can tighten up
  the range.

  The deviation of an interval is also a range.  We need to find
  out what that range is.  

  In OR and AND nodes, we store the number of closed chilren whose
  value is 1.  Then 
  The value of an OR node is [x, y], where x is 1 if some child
  has value [1, 1] and 0 otherwise, and y is 0 if every child has
  value [0, 0] and 1 otherwise.  We need to store in the OR node
  the number C of children whose min value is 1.  Then the value is
  [(C > 0 ? 1 : 0), 1].  We could tighten things up 1 by storing
  the number of children D whose max value is 1, and then the value
  is [(C > 0 ? 1 : 0), (D > 0 ? 1 : 0)].  But we just want to store
  a single integer in the signature, so we don't do the second thing.

  The value of an AND node is [x, y], where x is 1 if every child
  has value [1, 1] and 0 otherwise, and y is 0 if some child has
  value [0, 0] and 1 otherwise.  Again, if we store in the AND
  node the number C of children whose min value is 1, then the
  value is [0, 

  I've written a new section of the theory appendix, entitled
  "Value sum expressions", which generalizes counter monitors
  into value sum expressions.  It's all done except for dominance
  testing, which is a black hole at the moment.  I'm expecting
  to have to fall back on tradeoff dominance, or even strong
  dominance, but it's all still to do.

8 February 2024.  I've documented a VAL_SUM type that looks like
  it will do the job.  It can deal with int-valued or float-valued
  children, but by always including a deviation calculation it
  always produces an int value.  It optionally includes a cost
  calculation.  When present it makes dominance testing easier,
  when absent we have to fall back on strong dominance or some
  such thing.

  Even when a cost is present there is some question over how
  dominance testing works in practice.  Really, dominance testing
  needs a complete clean-out.

9 February 2024.  Starting to implement the revisions to dynamic
  today.  I've decided to start by introducing a NUMBER type
  which is an untagged union of int and float.  Then I will copy
  the INT_SUM_COST code, converting the type name to VAL_SUM, and
  take it from there.  Fingers crossed.

  Actually I already have that untagged union, it's called
  KHE_DRS_VALUE.  I'll stick with that but use it more.  I've made
  the elements of each signature have type KHE_DRS_VALUE, and this
  has allowed me to remove the FloatToInt and IntToFloat functions
  that I was using previously to encode floats as ints (actually
  within shorts, which are too short really) within signatures.

  Added type KHE_DRS_EXPR_SUM, which is what I have been calling
  VAL_SUM until now.  Currently it's a clone of INT_SUM_COST.  All
  the boilerplate is done, ready for the new stuff.

10 February 2024.  Now building limit busy times and limit workload
  expression trees.  Including replacing WORK_TIME by WORK_DAY where
  appropriate.  I've also added value upper bounds, including
  cumulative value upper bounds.  All audited with clean compile.

11 February 2024.  Broke EXPR_SUM into EXPR_SUM_INT and EXPR_SUM_FLOAT.
  Done the basic boilerplate, checked the types of all fields, and now
  I have a clean compile.  Also replaced KheDrsOpenChildrenAtOrAfter by
  KheDrsExprSumIntChildrenAtOrAfterUpperBound and
  KheDrsExprSumFloatChildrenAtOrAfterUpperBound.  Also moved the
  reverse cumulative upper bound arrays into EXPR_SUM_INT and
  EXPR_SUM_FLOAT, since only they use it.

12 February 2024.  Thinking about dom tests, no code written today.
  A straw in the wind:  KheDrsExprIntSumCostChildDomTest and
  KheDrsExprIntSeqCostChildDomTest are the same except that
  KheDrsExprIntSumCostChildDomTest has corr_dom_table4.

13 February 2024.  Removed KHE_DRS_EXPR_INT_SUM, KHE_DRS_EXPR_FLOAT_SUM,
  KHE_DRS_EXPR_INT_DEV, and KHE_DRS_EXPR_FLOAT_DEV.

  I seem to have corrupted the file somehow, presumably by clicking
  on the mouse wheel when I didn't mean to.  So I wasted a lot of
  time fixing that up.

  Untangled the dom test code without changing its semantics, by
  moving to a single KheDrsExprDomTest with the usual big switch.
  Made a separate dom test submodule for each expression type,
  to delimit the chaos.  Everything is neat now, ready to audit.

  Expanded KheDrsExprInitEnd so that it has a type switch.  But
  I haven't yet added to the different cases the code that they
  need, notably adjusting limits.

14 February 2024.  Converted KheDrsExprInitEnd to strongly
  typed, and moved all end of initialization stuff to the
  strongly typed versions.  All good, all done and dusted.

  Added AdjustLimits code to SUM_INT and SUM_FLOAT.

15 February 2024.  Posted a new version (Version 2.10) today
  in response to a reminder from a correspondent.  I've had
  to turn off optimal reassignment using dynamic programming
  but my correspondents at the moment are all high school
  timetablers so that's OK.

  It's the open children whose value upper bounds I need to
  accumulate, not all children.  KheDrsOpenChildrenSetUpperBounds
  written and called, and KheDrsOpenChildrenUpperBoundsAtOrAfterInt
  and KheDrsOpenChildrenUpperBoundsAtOrAfterFloat written, ready
  to be used.

16 February 2024.  Now using KheDrsOpenChildrenUpperBoundsAtOrAfterInt
  and KheDrsOpenChildrenUpperBoundsAtOrAfterFloat, but those uses
  need a careful audit.

  Tidied up all the expression make functions, including adding
  a value_ub paramter to KheDrsExprInitBegin so that all of the
  common attributes except value are initialized by it; value is
  initialized by KheDrsExprDoInitEnd.

17 February 2024.  I've changed the open children code so that
  it holds the sum of the upper bounds of the open children even
  while children are still being opened.  But it does not hold
  the cumulative sums until the expression itself is opened.

  Went on a little algebra excusion, showing that the maximum value
  of an expression whose value is a deviation is

      allow_zero ? max(0, L - 1, M - U) : max(L, M - U)

  where M is an upper bound on the determinant.  The L
  case arises when the determinant is 0, the M - U case
  arises when M >= U and the determinant is M.

  Looking through the code, it seems that everyone needs a
  value_ub field.  So the question is, when should it be
  initialized?  And the answer is by the time the expression
  has been completely built - by InitEnd.  So why not do it
  all in InitEnd?  Yes, let's do that.  All done and dusted
  now.  I've also implemented the "algebra excursion" above.

  I've now sorted out all the expression initialization,
  including value_ub.  It would be good to document it,
  but not now while I'm on a roll.

19 February 2024.  Changed the open children submodule to a
  new version that keeps everything up to date as children
  are added and deleted.  I have a clean compile and I've
  done a careful audit.

20 February 2024.  Sorted out KheDrsExprIntSumCostUpdateLU.  The
  old code was correct.  I've added some commentary to explain why.
  Finished auditing INT_SUM_COST and INT_SEQ_COST, including
  removing eisc->first_open_child_index:

    /* find the index of child_e in eisc->open_children */
    open_index = eisc->first_open_child_index;
    HnAssert(HaArray(eisc->open_children_by_day.open_children, open_index)
      == child_e, "KheDrsExprIntSeqCostChildHasClosed internal error 1");
    eisc->first_open_child_index++;

  I guess open_index is always 0 now and first_open_child_index
  is no longer needed.  KheDrsExprIntSeqCostChildHasClosed did
  end with this:

    HaArrayPut(eisc->closed_seqs, open_index + 1, NULL);

  but now I'm deleting that whole position.

  Audited SUM_INT and SUM_FLOAT.  All done except I need to
  think about KheDrsAdjustedSigVal.

  The PATAT paper deadline is 15 March, but I'm going bushwalking,
  so the deadline for me is Wednesday 6 March - just two weeks away.

21 February 2024.  Removed KheDrsAdjustedSigVal from SUM_INT and
  SUM_FLOAT.  Careful thought might allow it to be reinstated,
  but there are problems if we're not at the root, and problems
  when the value is a float, so I'm keeping away from it.

  Now looking at what should be int and what should be float in
  limit workload constraints.  Everything going into ld and ud
  can and so probably should be float.  There is no history but
  if there was it would apply to each time group and so it too
  would be float.

  I'm currently storing min_limit and max_limit as floats, but
  that seems wrong since they are ints in the constraint.  But
  it does no harm because they get compared with floats.  But
  the dom tests expect ints.  So the conclusion is:  all is well,
  unless I find something different when I come to dom tests.

  Making a start on understanding dominance testing, by reviewing
  it for OR expressions.  I really need to rebuild my understanding
  of strong dominance.

22 February 2024.  I seem to have stumbled into rewriting (or rather
  reorganizing) the dominance testing documentation.  So I'm going on
  with that.  Hopefully it will be finished today or tomorrow, and I
  can then go on and review the code.  I've finished Separate and
  Tradeoff dominance sections, and started Tabulated.

23 February 2024.  Still rewriting the dominance documentation.
  I've more or less got the structure right now.

24 February 2024.  Going through the theory appendix from start to
  finish, bringing everything into good order.  I've done it all
  now, except solution types, which don't need a review just now.

  Updated the code and doc, replacing "weak" by "equality",
  "strong" by "separate", and "uniform" by "tabulated".

25 February 2024.  Starting to review the implementation of
  SUM_INT and SUM_FLOAT dominance testing today.

  Decided to remove AdjustLimits:  it doesn't really help in
  any way, it just complicates things by making it harder
  to see whether there is a maximum or minimum limit.

  Removed min_limit and max_limit from dom_test; they were
  unused, as it turns out.  But can't remove allow_zero, it
  is used, for example in the tilde function of separate
  dominance.  Here's who uses what:

      allow_zero
        tilde, KheDrsDomTestDominatesTradeoff,
	KheDrsDomTestDominatingSet, KheDrsDomTestDominatedSet,

      a
        KheDrsDomMax, KheDrsDomTestIsStrictEquality,
	KheDrsDomTestDominatingSet, KheDrsDomTestDominatedSet,

      b
        KheDrsDomMin, and rest as for a

      combined_weight
        KheDrsSignerCorr1Dominates, KheDrsSignerCorr2Dominates,
	KheDrsSignerPsiGet, KheDrsTryTradeoff,
	KheDrsDomTestDominatesTabulated, also trie code

      main_dom_table2
        KheDrsExprIntSumCostDoDomTest, KheDrsExprIntSeqCostDomTest

      corr_dom_table4
        KheDrsExprOrIntSumCostDomTest

  Also we have drs->solve_dom_test_type, which could be

    KHE_DRS_DOM_TEST_UNUSED
    KHE_DRS_DOM_TEST_SEPARATE
    KHE_DRS_DOM_TEST_TRADEOFF
    KHE_DRS_DOM_TEST_TABULATED

  So these are the dom test types we actually implement,
  plus correlated.

  Basically, all we need to do is understand a and b, and
  then we can implement separate dominance for the new
  types, and then we're done.  We need to document the
  choice of a and b for the various expression types;
  after all they all support separate dominance.

  I've documented how to choose a and b for INT_SUM_COST
  dominance tests, and reviewed the implementation so that
  it follows the documentation.

26 February 2024.  I've reviewed the documentation and implementation
  of separate dominance for INT_SUM_COST, INT_SEQ_COST, OR, and AND
  expressions.  And now I've written the documentation for SUM_INT
  and SUM_FLOAT.  It needs an audit, then I'll be ready to implement.

27 February 2024.  Finished auditing the dominance documentation and
  the diagrams.  Also finished revising the construction code.  Made
  sure that SUM_INT and SUM_FLOAT construction does not call
  KheDrsExprCostSetConstraint.

  Revised KheDomTestTypeActual and checked check all its uses;
  tradeoff_allowed now means "tradeoff or tabulated allowed".
  Also implemented the SUM_INT and SUM_FLOAT dominance tests
  in accordance with the documentation.

28 February 2024.  Audited yesterday's stuff.  All good.

  Sorted out some confusion over which trees to build for
  prefer resources monitors; the confusion was over when
  they are already taken account of by ass_cost attributes.
  All done and documented now.

  There are cases where limit resource monitors mimic prefer
  resources monitors.  In those cases we have to ignore them.
  All done and documented now.

29 February 2024.  Yep, it's a leap year.  Finished removing tries
  (not KHE_DRS_SHIFT_SOLN_TRIE, it's indexed by a resource set).
  Converted INT_SUM_COST to COUNTER and INT_SEQ_COST to SEQUENCE.

1 March 2024.  Testing today.  Wasted most of the morning chasing a
  bug which turned out to be a debug function being called when the
  solution is a placeholder, which it was not suited for.  Grrr.
  But subsequently fixed a few other small bugs, including forgetting
  to remove the open child of a SEQUENCE expression; finding that
  I was initializing drs->max_workload_per_time when opening, when
  I should have been doing it when making the solver; and storing
  a WORK_DAY value into value.i instead of value.f.  I now have
  a full run of 12 parallel solves of COI-SINTEF:

    [ "COI-SINTEF", 12 threads, 12 solves, 8 distinct costs, 6.2 secs:
      0.00001 0.00002 0.00003 0.00004 0.00005 0.00005 0.00006 0.00006
      0.00007 0.00007 0.00007 0.00010
    ] best soln (cost 0.00001) has diversifier 0

  There are cases where I'm finding an improvement, the best being

    KheDynamicResourceSolverDoSolve ret. true (new 0.00002 < old 0.00003)

  So that pretty well proves that things are running as intended.
  And just by trying more dynamics before giving up, I got this:

    [ "COI-SINTEF", 12 threads, 12 solves, 6 distinct costs, 6.6 secs:
      0.00000 0.00001 0.00001 0.00001 0.00002 0.00002 0.00002 0.00002
      0.00003 0.00004 0.00004 0.00005
    ]

  So that's the result I was looking for.

  Tried all of COI, got an inconsistent monitors in COI-QMC-2.xml.
  So I'm looking into that now.  PRC:A=s1:1:RG:Empty/2Thu:N/A=s1:1
  is one of them.  I need to do a rerun print of it.  I've set
  up doit, now I need to do the debug flags in dynamic.

2 March 2024.  Carrying on with testing.  The current problem is
  a disagreement between KHE cost and packed soln cost, but no
  individual monitors disagree.  I have some debug output that
  shows that everything is in agreement except that these two
  monitors:

    [ A1 00261 PRC:A=s1:1:RG:Empty/1Tue:N/A=s1:1     0.00001 ]
    [ A1 00291 PRC:A=s1:1:RG:Empty/3Sun:L/A=s1:1     0.00001 ]

  are KHE defects but not dyn defects.  But they should not be
  dyn defects because they are reported via asst_cost and not
  included as dyn monitors at all.  But the problem is, the
  total cost of dyn monitors is 31, and yet the packed soln
  cost is 34, suggesting that there have been three additions
  of 1 instead of the required 2, but my debug output does not
  show any.  So it's quite a tangle.  I guess the first step
  is to get some proof that asst_costs are being added.
  Actually the assignments are to resources Q and P, and these
  are not being unassigned at all.  So that extra 2 should be
  part of the initial cost.

3 March 2024.  Carrying on with testing.  Fixed yesterday's bug
  first time, following an idea that came to me in the middle of
  the night, which was to take account of dt->ass_cost when
  unassigning tasks while opening resources.  Got this:

    [ "COI-QMC-2", 12 threads, 12 solves, 4 distinct costs, 19.5 secs:
      0.00031 0.00031 0.00031 0.00032 0.00032 0.00032 0.00032 0.00032
      0.00032 0.00034 0.00034 0.00035
    ]

  The paper run (which has a longer time limit) gets down to 30,
  only one more than the optimal cost, 29.  COI-QMC-1 has also improved,
  from 18 to 16 (optimal is 13).

  If parallel solve finds a solution that matches the lower bound (or
  just 0), end the parallel solve early.  Done and documented.

4 March 2024.  Still testing.  I'm well into INRC2 and getting zero
  bugs, but the INRC2-8 results all contain a few infeasibilities.
  I think the problem might be that grouping leads to infeasibilities
  and there is not enough time at the end of a five minute run to
  get rid of them.  Here is a 30-minute run:

    [ "INRC2-8-030-1-27093606", 1 solution, in 30.0 mins: cost 0.02545 ]

  This is 20% worse than the LOR solution, which has cost 2125.
  Got 3265 in five minutes by deleting grouping.

7 March 2024.  A big day for getting things finished.  I submitted
  my KHE24 paper to PATAT 2024 today.  I also posted it and its
  supporting data files on my web site.  I also posted the current
  version of KHE (denominated Version 3.11) on my web site today.

17 March 2024.  I've been away, work starts again today.

18 March 2024.  Doing some runs to throw up the expand_by_shifts bug,
  but no luck so far.  I may just go to work on the usual stuff and
  wait for it to appear again.

19 March 2024.  Decided to leave the KheDrsTaskClose for a while.
  Added a -o flag to khe and documented the out_file_name parameter
  of KheArchiveParallelSolve.  Implementing it is next.

20 March 2024.  Decided overnight to change the interface so that
  solution caching is internalized into the parallel solver, even
  at the cost of a (rare) double read.  Just makes more sense.

21 March 2024.  Finished implementing solution caching.  It needs
  an audit, then it will be ready to test.

22 March 2024.  Audited solution caching.  It's ready to test.

23 March 2024.  Testing solution caching.  Fixed a couple of
  tiny bugs, now it seems to be working well.

  Going to work on INRC2-4-030-1-6291 again.  Here is my
  previous result from 14 January 2024:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 10 dist costs, 299.3 secs:
      0.01845 0.01890 0.01900 0.01900 0.01940 0.01950 0.01970 0.01970
      0.01990 0.01995 0.02010 0.02095
    ] best soln (cost 0.01845) has diversifier 8

  And here is my best result so far, from 12 March 2023:

    [ "INRC2-4-030-1-6291", 4 threads, 8 solves, 7 distinct costs, 28.8 mins:
      0.01785 0.01840 0.01845 0.01865 0.01875 0.01875 0.01890 0.01925
    ]

  Although it seems to be a fluke (look at the second best, 1840).  A
  more realistic number to aim for seems to be 1810 or 1820.  But what I
  am actually getting today is

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 10 distinct costs, 5.0 mins:
      0.02015 0.02040 0.02045 0.02045 0.02065 0.02070 0.02105 0.02110
      0.02110 0.02115 0.02140 0.02160
    ]

  I need to do something about this before I move on to longer runs.
  Here's another run with the same parameters, it seems to be better:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 11 distinct costs, 5.0 mins:
      0.01955 0.02000 0.02015 0.02040 0.02040 0.02045 0.02070 0.02080
      0.02095 0.02160 0.02185 0.02275
    ]

  There is nothing obviously wrong so I've decided to try a 10 minute
  run to see whether that improves things:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 10 distinct costs, 10.0 mins:
      0.01880 0.01950 0.01950 0.01955 0.01955 0.01960 0.01980 0.01995
      0.02000 0.02010 0.02020 0.02045
    ]

  And here is a 30-minute run:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01860 0.01870 0.01880 0.01900 0.01915 0.01920 0.01925 0.01925
      0.01935 0.01945 0.01980 0.01995
    ]

  Uninspiring, we need to look into who is underperforming.

25 March 2024.  Working on old friend INRC2-4-030-1-6291 today.
  Trying 5-minute solves and looking to see what the problem is.
  Here is what we are getting with the default solver (5 mins):

    [ "INRC2-4-030-1-6291", 1 solution, in 5.0 mins: cost 0.02095 ]

  This is currently running with a separate matching throughout.
  Taking away this matching gives this:

    [ "INRC2-4-030-1-6291", 1 solution, in 5.0 mins: cost 0.01870 ]

  Bingo.  Here is best of 24 with a 5 minute limit (2.5 minutes per solve):

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 17 distinct costs, 5.0 mins:
      0.01880 0.01895 0.01895 0.01905 0.01910 0.01915 0.01920 0.01930
      0.01930 0.01930 0.01935 0.01940 0.01940 0.01945 0.01945 0.01955
      0.01960 0.01970 0.01970 0.01980 0.01980 0.01995 0.02045 0.02060
    ]

  In the conference paper the result for this setting is 2040.  Now
  the same again but taking away the resource assignment invariant
  as well.  It should make no difference if there is no matching.

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 17 distinct costs, 5.0 mins:
      0.01885 0.01895 0.01905 0.01910 0.01910 0.01915 0.01915 0.01920
      0.01930 0.01930 0.01930 0.01935 0.01940 0.01950 0.01960 0.01960
      0.01960 0.01960 0.01965 0.01970 0.01995 0.02035 0.02040 0.02045
    ]

  Just a random difference.  And here is a run without a call on rrm
  that does not seem to be likely to be useful:

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 18 distinct costs, 5.0 mins:
      0.01890 0.01895 0.01895 0.01895 0.01915 0.01915 0.01935 0.01950
      0.01955 0.01960 0.01965 0.01965 0.01970 0.01970 0.01975 0.01975
      0.01980 0.01990 0.01995 0.02000 0.02020 0.02025 0.02040 0.02080
    ]

  Actually it was somewhat useful so we'll keep it in.  1880 is
  right on the desired 10% mark.  Here is best of 24 with a 60 minute
  limit (30 minutes per solve):

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 18 distinct costs, 60.0 mins:
      0.01770 0.01790 0.01810 0.01810 0.01820 0.01830 0.01845 0.01855
      0.01860 0.01860 0.01865 0.01870 0.01875 0.01875 0.01880 0.01885
      0.01885 0.01885 0.01890 0.01905 0.01905 0.01915 0.01920 0.01955
    ]

  In the conference paper the result for this setting is 1865.  And
  according to what I've written on 23 March 2024, this 1770 is a
  new best.  It is also at 1.04, so successful in practice.

26 March 2024.  Worked on the bug I found yesterday when testing ERMGH,
  and fixed it (around line 16139).  But the question is, do I need to
  revise the code in general to make it crystal clear that the signer
  code and the signature code are consistent with each other?

27 March 2024.  Done some tidying up of the fields of signers, a run
  revealed no problems with it.  Then removed the next_di parameter
  from KheDrsExprEvalSignature; you can get it now from the signer.

28 March 2024.  I'm now using KheDrsExprEvalType in
  KheDrsExprNotifyResourceSigners and KheDrsExprCounterNotifyCoverSigners.
  It needs and audit and then I'll be ready for testing.

29 March 2024.  Audited the revised KheDrsExprNotifyResourceSigners
  and KheDrsExprCounterNotifyCoverSigners.  Ready to test.

  Took me all morning to fix a bug that appeared when I was solving
  COI-Azaiez.  It was a stupid thing in new function KheDrsExprEvalType,
  where I was calling KheDrsOpenChildrenWithIndex when I should have
  been calling KheDrsOpenChildrenIndexInRange.

  Now sprung a bug in COI-ERRVH:

    KheDrsDim3TableGet: index 50 out of range 0 .. 39

  Limit resources monitor DemandConstraint:3A/EGDC:170 requires exactly 12
  Skill-5 nurses in event set EGDC:170, which contains events

    1Tue:E 1Tue:D19 1Tue:D22 1Tue:D23 1Tue:D12

  The expression has 65 children.  The closed state is 3, which is correct:
  I've checked that 3 of the children are assigned Skill-5 nurses.

30 March 2024.  Fixed the problem with child_count etc. in constraints.
  The problem was that this statement (line 10274):

    child_count = HaArrayCount(ec->children);

  takes only one expression, ec, into account, whereas there can be many
  monitors derived from this constraint, with different numbers of children.
  I need the max of this value, but even worse, there are other points where
  ec is referenced in this function.  Here they are and what to do about them:

  * child_count = HaArrayCount(ec->children);
      Take the max over all ec.  But this may not be good enough because
      of line 18279 "lim = child_count + dc->max_history - e;".  Actually
      no, that's OK, it does not change the calculation, just the range
      of values it's done for.

  * KheDrsExprCounterFindAvailUnweightedCost
  * KheDrsExprCounterFindCorrAvailUnweightedCost
      These two are similar.  They reference ec->history_after, ec->allow_zero,
      ec->max_limit, ec->min_limit, and ec->cost_fn, but these depend only on
      the constraint so they are the same for all ec.

  * KheDrsExprCounterNeedsCorrTable
      This checks whether all the children are single day or not.  In principle
      this could vary from one ec to another.  I've ended up just oring together
      the results of this function for all expressions that come in.

  I've done a 5:0 run of the whole COI archive and nothing crashed.

  Turning now to work on Andreas' bugs.  The first problem is in instance 13.
  It springs line 6817 in khe_se_solvers.c, in a section labelled "old code,
  auditing it is still to do, but it seems OK".  I guess it's not OK.
  I've documented and implemented the appropriate change to the semantics
  of meet bounds and meets, allowing a meet bound to be added to a meet
  more than once.  Ditto for task bounds and tasks.  A test of instance
  13 did not crash.

  Added a merge function to khe so that we can merge all these files
  into a single archive for testing.  All done and working well.

  Added a KheMeetBoundDebug function.

1 April 2024.  I've worked out the problem with KheMeetSplitUnchecked.
  It splits the child at a moment when the parent is still unsplit.
  Things would work if the parent was already split, but they don't
  work because it is not yet split.  We are failing the invariant at
  a time when it should not be expected to hold.  When fixing this we
  need to be alert to kernel operations so that it can get undone correctly.

2 April 2024.  Checked over everything, installed the bug fix for the
  second of Andreas's bugs, tested it, then did a longer run of all 18
  of Andreas' instances, 30 minutes each.  No crashes.

  Decided to make the default behaviour during resource assignment be
  to install a separate matching for high school timetabling but not
  for nurse rostering.  I've documented and implemented a "!" syntax
  to add to do-it-yourself solving for selecting an id based on the
  model.  All done and tested.

  Working on getting better error messages out of do-it-yourself
  solving.  All done and tested and seems to be working well.

3 April 2024.  Looking into the COI-Millar-2.1.1 bug.  Tried with
  expand_by_shifts=false and the bug is still there.  So there
  goes my first theory about it.  But the bug turned out to come
  from an earlier call to

    KheClusterBusyTimesMonitorSetMinimum(Constraint:4/Nurse1, 1 -> 2)

  Dynamic was taking its minimum limit from the constraint, when it
  should have been taking it from the monitor.  All fixed.  I checked
  for other monitors that can change their limits but found none
  except for SetCeiling, which I'll leave alone for now.

  Trying quite a long run, 5 minutes per instance.  So far it has
  run through COI, INRC1-LM, INRC1-S, INRC2-4, and INRC2-8 without
  any crashes.  But some of the COI results seem quite poor.

4 April 2024.  Here are the COI results that got worse:

     Instance            Best    KHE24x24 March   KHE24x24 April
     -----------------------------------------------------------
     COI-Millar-2.1.1       0                 0             2000
     -----------------------------------------------------------

  Actually there are some worse, some better.  This one stands out.
  Fiddled with rcm and got COI-Millar-2.1.1 back to 0.

  Tried various long runs of INRC2-8-060-2-10340391, no crashes.
  So I'm deleting the bug report about this that I made previously.
  It might come back but in the meanwhile it seems to be gone.

5 April 2024.  Built a stub for the workload packing solver.  It's
  now ready to use.  My first step will be to investigate why there
  is no workload in the Avail column of HSEval for COI-WHPP.

  I've got some debug output from khe_avail.c.  It's looking as
  though I might have hacked up the workload limit stuff so that
  it affects only busy times, and broken it for workload proper.

6 April 2024.  Changed the interface of the avail solver to
  remove the awkwardly stateful KheAvailSolverSetResource.
  All documented, implemented, and used throughout KHE.

  Changed the documentation so that there is now a separate
  section for explaining how max workload is calculated.  This
  separateness will be paralleled in the code, eventually.

  Spent quite a lot of time tidying up khe_avail.c:  making
  submodules, making the functions do more conventional things
  (Make, Free, etc.) and giving them more conventional names.
  Also I've made it do a recalculation when the resource
  changes and when the resource type changes.  It's in pretty
  good shape now, except that I haven't started on the new code.

7 April 2024.  Done another audit of khe_avail.c.

  KheAvailSolverAddLimitWorkloadNodes seems to treat each time
  group independently.  That is wrong, surely?  No, it's fine
  in this application.  We're trying to keep cost to 0, and
  each time group has a maximum limit and contributes to cost
  if that limit is exceeded.

  Worked on avail doc, specifically tooling up for upper bounds
  on workload per time.

8 April 2024.  Still working on avail doc.  It's all done and
  audited, ready to implement.

9 April 2024.  Finished the final audit of the avail doc.  I'm
  currently implementing.  I've been right through, putting in
  event resource sets, and now I have a clean compile of those.
  Also removed preasst_info, using Sxx to work that out now.
  Audited AvailSolverFindBestBusyTimes, it looks good, and
  it does not count zero nodes twice.  Also changed the doc
  to make it clear that zero nodes get counted naturally.

10 April 2024.  Audited the workload doc, it's great now and
  ready to implement.

11 April 2024.  In the middle of a massive conversion to code that
  can handle int limits or float limits.  Have clean compile, need
  to audit what I've done and add what's missing (mainly workload).

12 April 2024.  Yesterday was massive.  Today I've audited it and
  finished it off.  The file contains 3401 lines now.  I've tested
  it, fixed one little bug, and now things are working pretty well.
  Let's call it done and move on.  I started on 5 April 2024, so
  it has take a week.  Longer than I expected but not too bad.

13 April 2024.  Back to workload packing today.  Done some
  boilerplate, I am now testing whether supply equals demand,
  and also whether the number of distinct workloads is right.

14 April 2024.  Extended the workload packing doc to present
  an algorithm for deciding which resources go to which event
  resources.  All implemented, which means the whole solver
  is implemented.  It needs an audit, then test.

15 April 2024.  Working on workload packing.  Audited the doc
  and expanded it a bit; it's great now.  Audited the code and
  changed a few things; it's great now as well.  Then I tested
  it, found one little bug, and after that it worked and found

    [ "COI-WHPP", 1 solution, in 6.7 secs: cost 0.00005 ]

  Just what I wanted.  Currently running all of COI; so far,
  WHPP is the only instance affected.

  Found this bug when running COI-ORTEC02:

    [ KheWorkloadPack(soln of COI-ORTEC02, options, Nurse, tbg)
      KheTimeSetMaxWorkload internal error
      ...
    ]

  I fixed it by allowing empty ersets to have a defined max
  workload, namely 0.0.  I've run two complete data sets now:
  COI and both INRC1 files.  So workload packing is now finished.

16 April 2024.  I spent the day looking at the Avail columns in
  the planning timetables for the COI instances.  Several problems
  have turned up, all of which I have managed to fix.  I also
  looked at some planning timetables for INRC1 solutions, they
  were fine too.  So I think we're done with availability and
  workload packing.  I started them on 5 April 2024.

  Started some high school testing, and got this on AU-BG-98:

    9# KheDrsSolveClose internal error:
      KHE soln cost 11.01103 != packed soln cost 11.01093

  The likely cause was that dynamic was simply ignoring avoid
  split assignments monitors, when what it needed to be doing
  was declining to run when they are present.  Fixed now.

  The solver was running dynamic when all event resources were
  preassigned.  I've fixed this, all done and documented.

17 April 2024.  Working on the KheDrsTaskClose bug.  I've
  reproduced it single-threaded in /home/jeff/tt/school/solve.
  The problem is:

  KheDrsTaskAssign returning false (on day TG:vr_5, Tek-1 already
    assigned Event809.1

      Task          open day        open day         |  closed day
                    vr_4            vr_5             |  vr_6
      -----------------------------------------------+-----------------
      Event809.1                   [Tek-1               Tek-1]
      Event854.1    [Tek-1          Tek-1]
      -----------------------------------------------+-----------------

  Task Event809.1 has a fixed value, it does not open.  Event854.1
  opens and assigns Tek-1 but then apparently no-one notices that
  there is a clash until too late, when we come to assign the task
  at the end of the run.  Complicated, needs careful thought.  It
  might require some lookahead, to rule out the assignment of Tek-1
  to Event854.1 on the first day, on the grounds that it cannot be
  carried through to the second day.  Yes, that would probably do it.
  Any issues on the first day?  Or later days, say if there is a
  preassignment?  We need to look ahead for every resource on every day.

18 April 2024.  I've broken KheDrsMTaskAcceptResourceBegin into two
  parts:  one called KheDrsMTaskSuitsFreeResource that checks that 
  a given free resource is suited to an mtask, in that it lies in
  the mtask's domain and is open on each day of the mtask, and
  another called KheDrsMTaskAcceptResourceBegin (still) that finds
  an unallocated task within the mtask.  Audited and tested on COI
  and XHSTT-2014; it seems to have fixed the problem.

  Also altered dynamic so that it exits without making a solver
  when all of its monitors have cost 0.  Some tests I ran
  showed that some (not all) optimal room reassignment was
  being avoided by this test.

19 April 2024.  Finished lots of tests:  COI, XHSTT-2014, INRC1,
  and INRC2.  I just need to try Kristallidis's own tests and
  then I will be ready to package up and post a new release and
  inform Kristallidis.

20 April 2024.  Finished testing, including Kristallidis's own
  tests, posted Version 2.12 of KHE, and notified Kristallidis.
  Finished the PATAT refereeing, so I'm free to return to KHE.

  Going back to my old friend, INRC2-4-030-1-6291.xml.  Legrain's
  result is 1695.  My best result so far, from 12 March 2023, is 1785.
  Here is the result of today's 5-minute run:

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 18 distinct costs, 5.0 mins:
      0.01880 0.01880 0.01895 0.01905 0.01910 0.01915 0.01925 0.01930
      0.01930 0.01930 0.01945 0.01945 0.01955 0.01960 0.01965 0.01970
      0.01970 0.01980 0.01980 0.01985 0.01995 0.02000 0.02035 0.02090
    ]

  This is right on the 1.10 target.  And here is the result of a
  60-minute run:

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 19 distinct costs, 60.0 mins:
      0.01770 0.01790 0.01810 0.01810 0.01820 0.01830 0.01845 0.01850
      0.01855 0.01865 0.01870 0.01875 0.01875 0.01880 0.01885 0.01885
      0.01890 0.01890 0.01905 0.01905 0.01915 0.01920 0.01955 0.01960
    ]

  And this is 1.04, and the second best is 1.05.  So once again we
  are on target.  I need to look for a less successful instance from
  INRC2-4 and work on that.

21 April 2024.  After looking over the paper I've decided to make
  INRC2-4-100-0-1108 my next test.  Like INRC2-4-030-1-6291 is has
  four weeks, but it has 100 nurses, many more than 30.  Legrain got
  1245 for it, I'm getting 1910, which is a bad 1.53 relative score:

    [ "INRC2-4-100-0-1108", 12 threads, 24 solves, 20 distinct costs, 5.0 mins:
      0.01910 0.01920 0.01995 0.01995 0.02005 0.02020 0.02040 0.02050
      0.02060 0.02075 0.02085 0.02085 0.02090 0.02100 0.02125 0.02185
      0.02195 0.02195 0.02215 0.02220 0.02220 0.02235 0.02305 0.02370
    ]

  What about half as many solves, each running for twice as long?

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 5.0 mins:
      0.01875 0.01875 0.01910 0.01925 0.01965 0.01985 0.02005 0.02010
      0.02025 0.02035 0.02045 0.02220
    ]

  Somewhat better, relative score is 1.50, still not very good.  What
  about a much longer run, 12 solves each running 30 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 12 distinct costs, 30.0 mins:
      0.01600 0.01640 0.01660 0.01665 0.01700 0.01725 0.01740 0.01750
      0.01760 0.01785 0.01795 0.01855
    ]

  This gives a relative score of 1.28, which is still poor but
  a lot better than what we had.

  I made an enhancement to yourself which reports the total
  improvement in cost of each algorithm, and the total time spent.
  Here is the output:

    [ DoItYourself rs:
      Item  Calls      Cost    Secs
      rts       2 829.86195     1.2
      rdv       4   0.00015  1122.3
      rrm       4   0.00340     0.8
      rec       4   0.01500   675.7
    ]

  It shows what I suspected, that rdv takes a lot of time and
  contributes very little.

  Looking through my solution to INRC2-4-100-0-1108, most of
  the overloads are by 2.  This is probably not an accident
  because of the minimum limit of 2 on early, day, and late
  shifts.  We need to check whether the right pairs of
  adjacent shifts are being moved.

  Another issue is that for full-time nurses, to get to 20
  busy days in four weeks we need a vitrually continuous
  plan of 5 days on and 2 days off, but that is not going to
  give the right number of weekends (at most 2) as it stands.
  HN_9 is a good example of what to do; HN_4 is a good example
  of what not to do (so avail = 2), and fixing it looks hard.

  I've been having a good look at the planning timetable, trying
  to find ways to shift load from the overloaded head nurses to
  the underloaded head nurses.  It isn't easy.  Take HN_14:  it's
  underloaded by 2, but there doesn't seem to be anywhere where
  you could just add 2 shifts without exceeding the weekend limit.
  In the LOR solution, HN_14 and HN_15 are underloaded by 2.  In
  my solution, HN_4, HN_14, and HN_15 are underloaded by 2.  So
  it seems likely that HN_4 is the only head nurse that could
  actually take on more load.  But in the LOR solution, HN_4 is
  working 4 weekends, in my solution only 2.  So no joy there.

22 April 2024.  Still comparing my solution with Legrain's, it's
  hard to see any dramatic differences.  Legrain's has more
  unassigned tasks, about the same number of assigned optional
  tasks, etc.  The big cost difference is due to fewer overloaded
  resources, but that does not seem to come from any one difference.
  It comes from a few more unassigned tasks, a few less assigned
  optional tasks, a few less underloaded resoures, nothing major.

  Legrain                                       	Inf. 	Obj.
  -------------------------------------------------------------------
  Assign Resource Constraint (5 points) 		   	150
  Avoid Unavailable Times Constraint (7 points) 	   	70
  Cluster Busy Times Constraint (28 points) 	   		950
  Limit Active Intervals Constraint (3 points) 	   		75
  -------------------------------------------------------------------
      Grand total (43 points) 	   				1245


  Kingston 						Inf. 	Obj.
  -------------------------------------------------------------------
  Assign Resource Constraint (1 point) 	   			30
  Avoid Unavailable Times Constraint (7 points) 	   	80
  Cluster Busy Times Constraint (36 points) 	   		1300
  Limit Active Intervals Constraint (8 points) 	   		195
  -------------------------------------------------------------------
      Grand total (52 points) 	   				1605

  Added some new code:  when there is an avoid unavailable times defect
  for resource r at the root, it tries an optimal reassignment of the
  complete timetables of r and r2, for each r2 /= r.  Got cost 1775
  in 10 minutes.  There were 15 successes for the new code.

  Now trying a 30-minute run with 12 parallel solves, with the new code:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01670 0.01680 0.01690 0.01700 0.01710 0.01710 0.01725 0.01760
      0.01825 0.01835 0.01845 0.01895
    ]

  The same run again, but with the new code turned off:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01590 0.01645 0.01675 0.01700 0.01720 0.01740 0.01750 0.01750
      0.01785 0.01825 0.01830 0.01850
    ]

  It seems pretty clear that we are better off without it.  Here
  we are without the new code but with es_full_widening_on=true:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01565 0.01635 0.01665 0.01695 0.01715 0.01730 0.01735 0.01760
      0.01760 0.01775 0.01825 0.01855
    ]

  So there's some improvement, relative is now 1.25.  Looks like larger
  moves are the go, so try es_full_widening_on=true es_swap_widening_max=24:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01665 0.01690 0.01730 0.01735 0.01740 0.01795 0.01800 0.01860
      0.01865 0.01875 0.01875 0.01895
    ]

  It's got worse, I wonder why.  Let try
  es_full_widening_on=true es_swap_widening_max=10:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 10 distinct costs, 30.0 mins:
      0.01630 0.01660 0.01670 0.01685 0.01690 0.01690 0.01720 0.01740
      0.01750 0.01760 0.01765 0.01765
    ]

  Inferior again.  Return to just es_full_widening_on=true, but this
  time visiting cluster time groups in priority order (i.e. time groups
  where the assignment is needless):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01615 0.01645 0.01650 0.01660 0.01665 0.01685 0.01685 0.01720
      0.01730 0.01740 0.01790 0.01860
    ]

  It's worse, presumably because it's slower, although it could also
  have something to do with excluding days.  Anyway I've removed it.

  What is really killing my solution is the number of overall workload
  overloads, which cost 20 each.  I have (1300 - 60) / 20 = 62 of those,
  Legrain has (950 - 90) / 20 = 43 of them.  So there is an excess of
  (62 - 43) = 19 * 20 = 380.  Here is where it's coming from:

                          Legrain   Kingston   Delta
      -------------------------------------------------------------
      Total overloads          42         60     -18 (should be 19)
      Total underloads         24         31       7
      Assigned optional        18         25       7
      Unassigned                5          1       4
      -------------------------------------------------------------

  So the 18 or 19 extras are coming partly from underloads, partly from
  assigned optionals, and partly from leaving fewer tasks unassigned.

  When we try to reduce workload, what order do we try it in?  Are
  we trying first to remove stuff that need not be assigned anyway?
  (Answer: No)  When we try mtasks that have more than the minimum
  number of assignments, are we shuffling the others? (Answer: Yes)

  Does optimal resource reassignment omit assigned optonals?  I
  guess it does - it unassigns everything and then finds the best
  reassignment, which could involve leaving out assigned optionals.

23 April 2024.  Between phases, let's check for tasks that can be
  unassigned with no cost.  I'm doing that at the start of each
  ejection chain run now.  Yesterday my best results were:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 11 distinct costs, 30.0 mins:
      0.01565 0.01635 0.01665 0.01695 0.01715 0.01730 0.01735 0.01760
      0.01760 0.01775 0.01825 0.01855
    ]

  This is all defaults except with es_full_widening_on=true.  Now
  here is that same run again but trying unassignments at the
  start of each ejection chain call:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 12 distinct costs, 30.0 mins:
      0.01530 0.01605 0.01630 0.01670 0.01675 0.01695 0.01705 0.01720
      0.01755 0.01760 0.01825 0.01855
    ]

  What do you know, it's better, relative to 1245 is 1.22.  Details:

                          Legrain   Kingston   Delta
      -------------------------------------------------------------
      Total overloads          42         57     -15
      Total underloads         24         25       1
      Assigned optional        18         27       9
      Unassigned                5          3       2
      -------------------------------------------------------------

24 April 2024.  Lost to a flu vaccine reaction.

25 April 2024.  Working on KHE_RESOURCE_SELECT_AVAIL in
  khe_sr_dynamic_vlsn.c.  All done, audited, and documented, and a
  short test seemed to work.  Here is a long test with 12 solves
  and a 30-minute time limit, so comparable to the tests above:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01545 0.01560 0.01585 0.01685 0.01685 0.01695 0.01725 0.01740
      0.01755 0.01755 0.01785 0.01840
    ]

  The best cost is worse, but the second best is better.  Hard to
  know what to think of this result.  Just to confirm, here we
  go without avail(3):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01530 0.01605 0.01630 0.01670 0.01675 0.01695 0.01695 0.01705
      0.01755 0.01755 0.01760 0.01845
    ]

  There is not enough here to say anything one way or the other.

  In my current solution, why not 4Fri-4Sun Nurse32 -> Nurse34.
  One shift would have to be unassigned, but a workload overload
  of 3 would be reduced to 0.  Should be a net gain.  No, Nurse34
  would then go from 1 to 2 busy weekends, which is an overload.

  Decided to try a longer run (12 solves, 60 minutes) with avail(3):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01445 0.01545 0.01545 0.01560 0.01600 0.01640 0.01640 0.01665
      0.01670 0.01685 0.01705 0.01750
    ] 

  This is easily the best so far, rel is 1.16.  Without avail(3):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01530 0.01560 0.01600 0.01615 0.01630 0.01635 0.01675 0.01695
      0.01705 0.01710 0.01730 0.01775
    ]

  So now we have some evidence that avail(3) is better.

26 April 2024.  Here is the result of a full 60-minute run of
  INRC2-4-030-1-6291 from 20 April 2024:

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 19 distinct costs, 60.0 mins:
      0.01770 0.01790 0.01810 0.01810 0.01820 0.01830 0.01845 0.01850
      0.01855 0.01865 0.01870 0.01875 0.01875 0.01880 0.01885 0.01885
      0.01890 0.01890 0.01905 0.01905 0.01915 0.01920 0.01955 0.01960
    ]

 Here is the same run, this time with avail(3):

    [ "INRC2-4-030-1-6291", 12 threads, 24 solves, 55.8 mins:
      0.01830 0.01830 0.01840 0.01840 0.01840 0.01850 0.01860 0.01860
      0.01865 0.01865 0.01875 0.01875 0.01880 0.01880 0.01880 0.01885
      0.01885 0.01900 0.01905 0.01910 0.01910 0.01910 0.01915 0.01970
    ]

  It's inferior.  There were 14 overloaded resources at the end, so
  plenty of scope for avail(3) to improve things.

     Variant        Leader          Followers (always != leader)
     ---------------------------------------------------------------
     choose         Any             Any
     avail          Overloaded      Underloaded && similar domain
     similar        Any             similar domain
     ---------------------------------------------------------------

  Added "similar" by unifying and generalizing "choose" and "avail".
  Written, audited, and documented, and ready to test.

  OK then, here is a 30-minute 12-solve test of INRC2-4-030-1-6291 with
  the default value for rs_drs_resources, which is choose(3):

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 30.0 mins:
      0.01790 0.01830 0.01860 0.01860 0.01865 0.01875 0.01880 0.01895
      0.01900 0.01900 0.01935 0.01960
    ]

  And here is the same test with rs_drs_resources="similar(3)":

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 30.0 mins:
      0.01790 0.01830 0.01840 0.01845 0.01855 0.01860 0.01860 0.01865
      0.01865 0.01880 0.01885 0.01960
    ]

  The results are very similar but similar(3) is better overall.
  Now for the same tests with INRC2-4-100-0-1108, choose(3):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01530 0.01560 0.01595 0.01660 0.01665 0.01675 0.01720 0.01725
      0.01735 0.01755 0.01845 0.01855
    ]

  and similar(3):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01530 0.01585 0.01605 0.01655 0.01660 0.01675 0.01720 0.01735
      0.01735 0.01745 0.01760 0.01845
    ]

  There is really nothing between these two, but similar is better
  in principle than choose, so I've made it the default.  The rel
  here is 1.22, not nearly good enough for a 30-minute run.

  Discovered to my horror that KheLimitActiveIntervalsAugment was
  not attempting to repair too-short intervals lying entirely
  within history.  It should have been trying to extend them
  at the right end.  It is doing that now.  Also updated the
  documentation of KheLimitActiveIntervalsMonitorDefectiveInterval
  to make clear what it returns when the defective interval lies
  entirely within history.

  Here is the similar(3) test again (it's the default now):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01520 0.01615 0.01655 0.01670 0.01675 0.01680 0.01685 0.01685
      0.01695 0.01750 0.01805 0.01855
    ]

  Yep, it's a new best.  And for 6291:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 30.0 mins:
      0.01810 0.01840 0.01840 0.01855 0.01885 0.01890 0.01895 0.01905
      0.01910 0.01910 0.01930 0.01940
    ]

  It's slightly worse, possibly, but nothing to worry about.

27 April 2024.  Working on a better repair for removing small
  active intervals.  Two new elements:

  (1) Instead of finding the interval that covers time group tg,
      we need the interval that covers time groups tg1 and tg2;

  (2) We need to be able to turn off expanding.

  All done, new functions are KheClearIntervalMultiRepair and
  KheDoClearInterval.  Audited and ready to test.

28 April 2024.  Testing the new small active intervals repair.
  Here is a 5-minute 1-solve run on INRC2-4-030-1-6291:

    [ "INRC2-4-030-1-6291", 1 solution, in 5.0 mins: cost 0.01845 ]

  Here is a 30-minute 12-solve run on INRC2-4-030-1-6291.

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 30.0 mins:
      0.01820 0.01825 0.01825 0.01835 0.01855 0.01860 0.01860 0.01860
      0.01870 0.01880 0.01885 0.01915
    ] 

  The best solution is slightly worse than the 1810 we got on
  25 April, but overall these solutions are clearly better.
  And here is a 30-minute 12-solve run on INRC2-4-100-0-1108:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01620 0.01650 0.01660 0.01670 0.01675 0.01690 0.01690 0.01695
      0.01700 0.01730 0.01815 0.01850
    ]

  We've lost the very good 1520 solution, but it was always an
  outlier.  So let's stick with the new repair.  My 1620 solution
  has three history too short defects, but the same three defects
  appear in the LOR solution.  So we seem to be doing as well as
  possible now in the history too short department.

  I tried a full COI 5-minute test, but I got a crash:

    parallel solve of COI-ERMGH: starting solve 1
    ...
    parallel solve of COI-ERMGH: starting solve 24 (last)
    Segmentation fault
    make: *** [makefile:66: KHE24-COI.xml] Error 139

  So fixing that is next.

  Parallel with all this I have been bringing the dynamic solver
  documentation up to date.  I've briefly reviewed the theory
  chapter; there seems to be nothing to do there.

29 April 2024.  Working on the yesterday's COI-ERMGH, and
  simulataneously on the dynamic solver documentation.

30 April 2024.  Looked over limit active intervals augment
  code and changed one or two things but did not convince
  myself that it was buggy.  Anyway it's done.  Doing more
  long tests (INRC2-4 is marginally better, but really
  marginal) and updating the dynmic doc at the same time.
  In the middle of expressions.

  Made a bit of a mess of repairing too-short intervals by
  deleting them altogether, but then I got on to something
  more reasonable that led to this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01820 0.01840 0.01885 0.01890 0.01920 0.01920 0.01940 0.01985
      0.01995 0.01995 0.02070 0.02135
    ]

  But I'm hoping to do better.

1 May 2024.  Tidied up the too-short limit active intervals
  augment.  Hopefully it does everything we really want now:
  a kernel interval covering the full too-short interval, plus
  widening.  Doing some tests now:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01780 0.01830 0.01875 0.01890 0.01930 0.01935 0.01970 0.01980
      0.01995 0.02020 0.02085 0.02150
    ]

  Not bad.  Relative to 1245 this is 1.42.  So still a long way to go.

  Done a long seroes of tests.  Reults are OK, not wonderful.
  So I'm back to the basic question again of how to do better.

  Here is a 30-minute 12-solve run of INRC2-4-100-0-1108:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01580 0.01585 0.01610 0.01665 0.01690 0.01700 0.01720 0.01720
      0.01730 0.01775 0.01800 0.01820
    ]

  Rel. is 1.26.  And here is the same run but with rdv turned off:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 17.1 mins:
      0.01565 0.01565 0.01595 0.01645 0.01650 0.01660 0.01670 0.01675
      0.01685 0.01710 0.01740 0.01795
    ]

  There you are, faster and better without rdv.  Rel is 1.25.
  Now let's try the same thing on COI-ERMGH.xml.  The paper
  has 1603.  Here is a 30-minute 12-solve run without rdv:

    [ "COI-ERMGH", 12 threads, 12 solves, 25.4 mins:
      0.00865 0.00867 0.00890 0.00905 0.00909 0.00918 0.00936 0.00952
      0.00960 0.00965 0.01010 0.01016
    ] 

  Best is 779, so Rel is 1.11 which is getting somewhere.
  Perhaps the way forward is to develop more complex repairs
  within ejection chains, for example splitting a move between
  two target resources.

  Finished a huge test, stored in 90_khe24.pdf, of the 90/10
  time split.  It does seem to be a good bet, but 95/05 might
  be even better.

3 May 2024.  After another look through the results of yesterday's
  huge test, I've decided to adopt the 90/10 split.  It's all
  documented in the paper.

  Here is a run of 12 solves for 5 minutes each:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01680 0.01740 0.01800 0.01815 0.01830 0.01860 0.01885 0.01895
      0.01900 0.01985 0.02080 0.02080
    ]

  Its rel is 1.34, a lot better than 24 solves for 2.5 minutes each,
  which produced cost 1790 for rel 1.43.  So I'll need to do a huge
  test of everything with 12 solves for 5 minutes each, but not now.

  Trying a huge test with 12 solves each with a 5-minute limit, as
  opposed to 24 solves each with a 2.5-minute limit.  The results
  will go into 12_khe24.pdf.

  In the huge test, I'm getting 1675 for INRC2-4-100-0-1108, which
  relative to 1245 is 1.34.  Still a long way to go, but progress.

  Comparing the new 12_khe24.pdf (12 5-minute solves) with the slightly
  older 90_khe24.pdf (24 2.5-minute solves), I find that COI and INRC1
  are basically the same except that the larger COI instances come
  out better, while INRC2 is a better on 4-week and a lot better on
  8-week.  So we'll keep 12 5-minute solves and drop 24 2.5-minute solves.

4 May 2024.  I think the way forward now is to work on the current
  best solution to INRC2-4-100-0-1108, the best of 12 5-minute solves:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01680 0.01740 0.01800 0.01815 0.01885 0.01900 0.01935 0.01940
      0.01950 0.01955 0.01975 0.02010
    ]

  and see if it can be improved.  Rel is 1.34.  But then when I
  ran it separately I got 1620, whose rel is 1.30.

  I think we need to go back to double moves.  i.e. when we
  swap tasks from r1 to r2, we also need to try swapping nearby
  tasks from r2 to r1.  Partly to even up workload, partly to
  avoid over-long sequences of busy days.  No, this is already
  being done by wide swaps.

  Result of whynot run was

    [ "INRC2-4-100-0-1108", 1 solution, in 5.5 mins: cost 0.01685 ]

  But now I've changed one occurrence of swap_widening_max to
  move_widening_max (apparently it was an error) and run the 12
  5-minute solves again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01650 0.01650 0.01660 0.01680 0.01705 0.01715 0.01725 0.01750
      0.01770 0.01770 0.01790 0.01810
    ]

  Rel is 1.32.  This is actually quite a lot better, in that the first
  three solutions are better than the previous best.  It suggests that
  more focus to the ejection chain repairs might be the way forward.

  Removed the from_mt parameter from KheSwapRepair, and am now
  verifying that both its resources are non-NULL.  I repeated the
  previous run and got similar results.  Also updated the doc,
  where it said that a swap resource could be NULL but never is.
  Now it says that a swap resource cannot be NULL.

5 May 2024.  Finished bringing the implementation appendix of the
  dynamic solver documentation up to date.

  Tried 12 60-minute solves of INRC2-4-100-0-1108:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01530 0.01570 0.01615 0.01620 0.01630 0.01630 0.01640 0.01645
      0.01660 0.01695 0.01715 0.01755
    ]

  This is significantly better; res = 1.22.  It suggests that
  anything that speeds up the existing code would help, but also
  that to get to rel of 1.10 and 1.05 new ideas will be needed.
  Another 12 5-minute solves, came out a bit different:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01625 0.01650 0.01650 0.01660 0.01715 0.01725 0.01745 0.01750
      0.01755 0.01760 0.01790 0.01850
    ]

  Another 12 5-minute solves, but with es_full_widening_on=true:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01615 0.01695 0.01730 0.01755 0.01760 0.01760 0.01775 0.01775
      0.01790 0.01795 0.01810 0.01855
    ] 

  It's come out slightly better, rel = 1.29, although the second
  best is worse.  Now trying with es_full_widening_on=true and
  max_beam=2:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01680 0.01780 0.01780 0.01805 0.01820 0.01830 0.01835 0.01840
      0.01855 0.01880 0.01890 0.01930
    ]

  This is evidently inferior, and it will be because it's slower.

  Designed and implemented KheDynamicResourceSequentialSolve.  It's
  all done and documented but it needs an audit.

6 May 2024.  Working on KheDynamicResourceSequentialSolve, which
  uses dynamic programming to make the initial assignment instead
  of time sweep.

  Had a bug, its calls on the solver all seem to be returning false,
  and yet I have debug output showing that solutions are being
  melded into the last day's solution set.  The problem is this:

    ending day 27 (4Sun), undominated 43, kept 43

  There are 43 solutions on the last day!  How can that be?
  So the test "KheDrsSolnListCount(soln_list) == 1" fails.  The
  problem is in KheDrsHardIndexedSolnSetRemoveDominated, which is
  logically wrong.  Also check KheDrsHardIndexedSolnSetDominates.
  I've fixed the bug now.

  Got a complete run (had to limit expansions per day to 500 to
  fit the time limit).  Here is the last line of debug output:

    KheDynamicResourceSequentialSolve returning true
      (init 169.42500. final 3.25380)

  I need to compare this 3.25380 with what time sweep ends with.
  The final cost is

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01700 ]
    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01685 ]

  It's not bad.  Best of 12:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01820 0.01855 0.01930 0.01965 0.01970 0.01980 0.01985 0.02020
      0.02210 0.02270 0.02350 0.02360
    ]

  Why is it worse than one?  Could it be memory contention?  Try again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01855 0.01860 0.01895 0.01965 0.01970 0.01975 0.01985 0.01990
      0.02250 0.02270 0.02350 0.02430
    ]

  This is to be compared with the result I got using time sweep:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01625 0.01650 0.01650 0.01660 0.01715 0.01725 0.01745 0.01750
      0.01755 0.01760 0.01790 0.01850
    ]

  All of the above was without sorting the resources.  I'm sorting
  them now and I get this from a single solve:

    KheDynamicResourceSequentialSolve returning true
      (init 169.39900. final 3.22015)

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01795 ]

  and this best of 12:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01750 0.01850 0.01855 0.01855 0.01875 0.01885 0.01895 0.01940
      0.02130 0.02160 0.02425 0.02485
    ]

  It seems to be inferior to time sweep.  Try one resource at a
  time; it will run faster, that might be better:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01795 0.01825 0.01835 0.01850 0.01855 0.01920 0.01925 0.01990
      0.02020 0.02030 0.02105 0.02165
    ]

  Actually it has come in rather worse.  Here we are trying three
  resources at a time, just for completeness (best of 12 and best of 1):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01655 0.01795 0.01830 0.01840 0.01840 0.01865 0.01875 0.01875
      0.01895 0.02000 0.02065 0.02090
    ]

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01740 ]

  Best so far, curiously.  So let's try four:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01860 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01755 0.01770 0.01780 0.01865 0.01875 0.01895 0.01925 0.01925
      0.01950 0.01975 0.02020 0.02045
    ]

  It all needs careful thought and analysis.  Not competitive right
  now, but maybe it just needs some tweaking.

7 May 2024.  What to conclude from yesterday's tests?  I really
  need to compare the solutions at the end of time sweep with
  the solutions at the end of dynamic sequential.

  Here is a single 5-minute solve using sequential dynamic:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 4.39060 ]

  It seems to have chewed up the full 5 minutes.  And here is
  time sweep:

    [ "INRC2-4-100-0-1108", 1 solution, in 1.2 secs: cost 0.03460 ]

  Much better in much less time.

  The problem is that a solution that assigns a compulsory task
  will always beat one that doesn't, and so the first resources
  get filled up completely - which is mad.  So let's reduce the
  cost of hard cover constraints to soft 2, and the cost of
  soft cover constraints to soft 1, and try again.  Do it in
  the KHE solution, not just in DRS.  But if we do that, there
  is almost no point in assigning two or more resources at the same time.

  NB On instances longer than 28 days, there will be no incentive
  to get things right in the first half for the second half.
  But this is no worse than time sweep in the same situation.

8 May 2024.  Finished replacing all calls to KheConstraintCost
  by calls to KheMonitorDevToCost.  Also initializing the new
  cost_function and combined_weight fields.  So the monitor
  functions for setting and using combined weight are all done.
  Updated KheSolnEnsureOfficialCost to ensure that original
  combined weights are used in all monitors.  Updated
  KheSetMonitorMultipliers to use combined weights rather
  than multipliers.  All done and documented.

  Used the new KheMonitorSetCombinedWeight function to
  implement a monitor adjuster submodule of sequential
  which adjusts event resource monitor weights.

  Got rid of KheConstraintCombinedWeight and KheConstraintCostFunction
  from solvers, as far as possible.  These are dangerous, especially
  KheConstraintCombinedWeight, because the weight could change
  before some solver begins.

  First results from sequential, one resource at a time:

    [ "INRC2-4-100-0-1108", 1 solution, in 4.6 secs: cost 0.03800 ]

  It is inferior to time sweep in both cost and running time; but
  not desperately inferior.  Worth further effort.

  Tried doing it 2 resources at a time.  It was a lot slower,
  and somehow the cost has come out worse:

    [ "INRC2-4-100-0-1108", 1 solution, in 86.2 secs: cost 0.04330 ]

  Now I've added randomness to the order that equally available
  resources are assigned by Sequential.  The 5-minute runs now give

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 5.0 mins:
      0.01820 0.01825 0.01835 0.01890 0.01895 0.01910 0.01915 0.01920
      0.01920 0.01925 0.01940 0.01955
    ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01645 0.01645 0.01680 0.01685 0.01685 0.01740 0.01750 0.01790
      0.01795 0.01860 0.01865 0.01885
    ]

  Here they are for time sweep:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 5.0 mins:
      0.01820 0.01855 0.01860 0.01860 0.01865 0.01880 0.01895 0.01900
      0.01910 0.01930 0.01935 0.01990
    ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01625 0.01650 0.01660 0.01710 0.01715 0.01715 0.01725 0.01755
      0.01770 0.01775 0.01790 0.01810
    ]

  There is really nothing between these results.  Legrain's result
  for INRC2-4-030-1-6291 is 1695, for INRC2-4-100-0-1108 is 1245.

                         Legrain      Time Sweep    Sequential
    -----------------------------------------------------------
    INRC2-4-030-1-6291   1695 (1.0)   1820 (1.07)   1820 (1.07)
    INRC2-4-100-0-1108   1245 (1.0)   1625 (1.30)   1645 (1.32)
    -----------------------------------------------------------

9 May 2024.  Added unavailable times to the criteria for sorting
  the resources in Sequential.  Result for INRC2-4-100-0-1108 is

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01640 0.01645 0.01680 0.01685 0.01685 0.01715 0.01740 0.01825
      0.01825 0.01825 0.01855 0.01885
    ]

  It is marginally better but nothing dramatic.

10 May 2024.  Going back to time sweep until such time (if ever)
  that I dream up an improvement to sequential.

                         Legrain          KHE
    ------------------------------------------------------
    Total underloads          24           19
    Total overloads           43           58
    Unnecessary assignments   16           31
    ------------------------------------------------------

  And 43 * 20 = 860, 58 * 20 = 1160.  This explains the
  results we are getting, although you have to add a bit
  more for max working weekends defects.  So the problem I
  really need to focus on is getting rid of unnecessary
  assignments.  I could save (31 - 16) * 20 = 300 if I
  could get them down to Legrain's level.  That would be
  cost 1640 - 300 - 1340 which is rel = 1.07.

  Look at 3Sat/3Sun.  The number of busy resources on those
  two days is

                         Legrain          KHE
    ------------------------------------------------------
    Busy 3Sat/3Sun            29           33
    Unassigned 3Sat/3Sun       1            0
    Unnecessary 3Sat/3Sun      3           10
    ------------------------------------------------------

  So there are (33 - 29) * 2 = 8 unnecessary shifts being
  worked on those two days in the KHE solution.  I've
  verified that the KHE solution has 10 unnecessary shifts
  and the Legrain solution has 3 unnecessary shifts.

  Trying unassigning tasks that do not need assignment when
  they are at either end of a sequence of task moves and
  they would otherwise be assigned to some resource.  This
  is because my solutions assign too many tasks that do
  not need assignment, and I'm looking for a way to tip
  the balance against such assignments.  I stupidly called
  KheMTaskNeedsAssignment when what I really wanted was
  KheMTaskContainsNeedlessAssignment.  (Are there other places
  where I call KheMTaskNeedsAssignment when what I really want
  is KheMTaskContainsNeedlessAssignment?  No.)  First results:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01585 0.01635 0.01675 0.01705 0.01710 0.01725 0.01770 0.01780
      0.01790 0.01850 0.01855 0.01930
    ]

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (2 points) 	   	  60
    Avoid Unavailable Times Constraint (5 points) 	  50
    Cluster Busy Times Constraint (41 points) 	   	1280
    Limit Active Intervals Constraint (9 points) 	 195
    --------------------------------------------------------
      Grand total (57 points)				1585

  What do you know, it's a new best, with rel = 1.27.  Even the second
  best is better than what I was getting before.  Here is the previous
  result, got without using KheMTaskContainsNeedlessAssignment:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01625 0.01680 0.01685 0.01690 0.01710 0.01715 0.01735 0.01745
      0.01770 0.01770 0.01790 0.01810
    ]

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (2 points) 	   	  60
    Avoid Unavailable Times Constraint (5 points)      	  50
    Cluster Busy Times Constraint (40 points) 	        1230
    Limit Active Intervals Constraint (11 points) 	 285
    --------------------------------------------------------
      Grand total (58 points) 	   		   	1625

  I put the new code in with the hope that it would improve
  violations of workload limits.  Instead it has made them
  worse, but improved active intervals defects more.  Queer.

  Now printing useful statistics under planning timetables:  total
  busy times overload, total busy times underload, total duration
  of unnecessary task assignments.

  Now there are 28 italic entries in the KHE solution, whereas
  there are 19 in the Legrain solution.  So if each leads to
  an overload elsewhere, the total unnecessary cost is
  (28 - 19) * 20 = 180.  Looking at 3Sat/3Sun, the total
  in Legrain is 3 unnecessary plus one unassigned, in KHE
  it is 6 unnecessary plus 0 unassigned.  This is a much
  less dramatic difference than I had previously.
  
  Altogether a pretty successful day, I'm getting results like

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01585 0.01635 0.01675 0.01705 0.01710 0.01725 0.01770 0.01780
      0.01790 0.01850 0.01855 0.01930
    ]

  consistently now.  For INRC2-4-030-1-6291.xml I get

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 5.0 mins:
      0.01815 0.01855 0.01860 0.01865 0.01875 0.01890 0.01910 0.01925
      0.01950 0.01950 0.01960 0.01980
    ]

  This is marginally better than the 1820 I was getting previously.

  Here is a 60-minute run:

    [ "INRC2-4-030-1-6291", 12 threads, 12 solves, 60.0 mins:
      0.01800 0.01840 0.01845 0.01845 0.01860 0.01875 0.01880 0.01900
      0.01925 0.01940 0.01940 0.01960
    ]

  This is rel = 1800 / 1695 = 1.06, which is close to the 1.05 mark
  that we are aiming for.  For INRC2-4-100-0-1108 for 60 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01535 0.01570 0.01575 0.01615 0.01620 0.01630 0.01635 0.01670
      0.01690 0.01730 0.01770 0.01810
    ]

  Rel is 1.23, still a fair way off the mark.

  Looking at INRC2-4-100-0-1108, Legrain's summary is

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (5 points) 	   	 150
    Avoid Unavailable Times Constraint (7 points)    	  70
    Cluster Busy Times Constraint (28 points) 	   	 950
    Limit Active Intervals Constraint (3 points)    	  75
    --------------------------------------------------------
      Grand total (43 points) 	   			1245

  and my current summary (5 minutes) is

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (2 points) 	   	  60
    Avoid Unavailable Times Constraint (5 points)    	  50
    Cluster Busy Times Constraint (38 points) 	   	1240
    Limit Active Intervals Constraint (11 points)    	 240
    --------------------------------------------------------
      Grand total (56 points) 	   			1590 

  Rel is 1.27.  So this is my starting point now.

11 May 2024.  Current state of workload balance is

                                     Legrain         KHE
    --------------------------------------------------------
    U  Available times (positive)        24           22
    O  Available times (negative)        43           53
    Y  Unnecessary assignments           19           28
    X  Absent assignments                 5            2
    --------------------------------------------------------
    U - O + Y - X                        -5           -5

  Our analysis said that U - O + Y - X would be constant
  (it's the amount by which supply exceeds demand in the
  instance), and indeed it is.  Great.  The point is that
  every unit of O and X incurs a cost.  To minimize O and
  X, we have to minimize U and Y.  Getting rid of some
  unnecessary assignments would help.

  If I could reduce Y from 28 to 19, that would be a
  reduction in cost of (28 - 19) * 20 = 180, which would
  make my total cost 1410, which would be rel = 1.13.

  Why wasn't a complete swap of TR_86 and TR_87 tried, to
  get rid of the unavailable time for TR_87?  Looks easy.
  Probably because there was an unnecessary assignment at
  one end, and removing that made the swap unattractive.
  I fixed that and got this just now:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01595 0.01620 0.01690 0.01695 0.01695 0.01710 0.01730 0.01735
      0.01745 0.01750 0.01750 0.01930
    ]

  The fix seems to have fixed the TR_86/TR_87 whynot, but
  overall things got slightly worse.  Perhaps I should try
  with and without removing the end, if there is one.

                                     Legrain         KHE
    --------------------------------------------------------
    U  Available times (positive)        24           25
    O  Available times (negative)        43           53
    Y  Unnecessary assignments           19           25
    X  Absent assignments                 5            2
    --------------------------------------------------------
    U - O + Y - X                        -5           -5

  I took away the blocking_tg from the full timetabling swap
  and got this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01640 0.01675 0.01685 0.01735 0.01740 0.01740 0.01755 0.01755
      0.01770 0.01775 0.01855 0.01870
    ]

  I guess it means it takes more time than it's worth.  It did
  reduce the unavailable cost from 70 to 40 though.  So I think
  I'll leave it in, on the grounds that it might be good when
  there is more time for such things.

  In the 1640 solution I have this:

                                     Legrain         KHE
    --------------------------------------------------------
    1Sat/1Sun unnecessary (TR only)        6           5  
    3Sat/3Sun unnecessary (TR only)        4           5  
    --------------------------------------------------------

  So I'm just as good as Legrain on these weekends.  But I
  am still 6 behind on unnecessary assignments.  I've also
  had a blowout in limit active intervals defects.

  We might be able to prove that a 20 percent load can't start
  work for several days.  Min consecutive busy days is 2,
  total workload is 6, therefore there are at most 3 chunks
  of 2 shifts each.  Max separation is 7, so working from
  the back we get
  
         2 busy  7 busy  2 busy  7 free  2 busy  7 free

  Total is 2 + 7 + 2 + 7 + 2 + 7 = 27. so such a resource
  cannot work on the first day, and if it works on the
  second day, its busy/free pattern is completely fixed.

  Summary of current state
  ------------------------

  Legrain's summary:

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (5 points) 	   	 150
    Avoid Unavailable Times Constraint (7 points)    	  70
    Cluster Busy Times Constraint (28 points) 	   	 950
    Limit Active Intervals Constraint (3 points)    	  75
    --------------------------------------------------------
      Grand total (43 points) 	   			1245

  KHE's summary (12 solves, 5 mins):

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (2 points) 	   	  60
    Avoid Unavailable Times Constraint (4 points) 	  40
    Cluster Busy Times Constraint (38 points) 	   	1240
    Limit Active Intervals Constraint (13 points)    	 300
    --------------------------------------------------------
      Grand total (57 points) 	   			1640

  Rel is 1.31.  I'm beating Legrain on assign resource and avoid
  unavailable times defects, but I'm a long way behind on cluster
  busy times and limit active intervals defects.  If I could equal
  Legrain on the latter I would save (300 - 75) in cost, which
  would be 1415, which would be rel = 1.13.  As for total workload:

                                     Legrain         KHE
    --------------------------------------------------------
    U  Available times (positive)        24           25
    O  Available times (negative)        43           53
    Y  Unnecessary assignments           19           25
    X  Absent assignments                 5            2
    --------------------------------------------------------
    U - O + Y - X                        -5           -5

  I'm fine on U, it's Y that's killing me.  If I could equal Legrain
  on Y, that would save (25 - 19) * 20 = 120, which would be 1520,
  which would be rel = 1.22.  For example, on 3Wed, Khe has 3 italics
  (all Early), Legrain has one italic (a Day).

12 May 2024.  Decided to try a larger limit on the number of
  recursive ejection chains calls.  Turns out that the current
  number was 200, so I increased it to 300 and got this (5 mins):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01640 0.01660 0.01675 0.01675 0.01685 0.01730 0.01735 0.01735
      0.01755 0.01755 0.01880 0.01915
    ]

  It's not really any different, so I've returned to 200.

  Written some interesting documentation on resource rigidity.
  Also done the boilerplate for implementing the idea, in
  khe_sr_consec_solver.c.

13 May 2024.  Am now sorting the sequential resources by increasing
  non-rigidity then decreasing availability.  It looks good.  I've
  enhanced KheDynamicResourceSequentialSolve so that it now does
  an arbitrary fraction ("frac") of the resources by dynamic and
  the rest (1 - "frac") by time sweep.  All done and documented.
  Here are some runs showing the cost at the end of construction:

    frac = 0.0:
    [ "INRC2-4-100-0-1108", 1 solution, in 1.2 secs: cost 0.03460 ]

    frac = 0.1:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.1 secs: cost 0.03655 ]

    frac = 0.2:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.0 secs: cost 0.03305 ]

    frac = 0.3:
    [ "INRC2-4-100-0-1108", 1 solution, in 2.9 secs: cost 0.03465 ]

    frac = 0.4:
    [ "INRC2-4-100-0-1108", 1 solution, in 2.9 secs: cost 0.03765 ]

    frac = 0.5:
    [ "INRC2-4-100-0-1108", 1 solution, in 2.9 secs: cost 0.03695 ]

    frac = 0.6:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.0 secs: cost 0.03985 ]

    frac = 0.7:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.1 secs: cost 0.04165 ]

    frac = 0.8:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.2 secs: cost 1.04455 ]

    frac = 0.9:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.3 secs: cost 10.04520 ]

    frac = 1.0:
    [ "INRC2-4-100-0-1108", 1 solution, in 3.4 secs: cost 52.03335 ]

  Now for some longer, full solves (12 solves, 5 minutes):

    frac = 0.0:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01635 0.01650 0.01655 0.01675 0.01685 0.01735 0.01740 0.01740
      0.01755 0.01770 0.01870 0.01880
    ]

    frac = 0.1:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01640 0.01650 0.01660 0.01660 0.01675 0.01680 0.01705 0.01720
      0.01725 0.01765 0.01820 0.01865
    ]

    frac = 0.2:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01660 0.01675 0.01685 0.01690 0.01710 0.01745 0.01745 0.01750
      0.01795 0.01800 0.01800 0.01810
    ]

    frac = 0.3:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01625 0.01625 0.01645 0.01685 0.01705 0.01730 0.01730 0.01735
      0.01735 0.01740 0.01755 0.01760
    ]

    frac = 0.4:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01590 0.01605 0.01655 0.01655 0.01665 0.01665 0.01700 0.01705
      0.01720 0.01790 0.01810 0.01860
    ]

    frac = 0.5:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01600 0.01610 0.01620 0.01635 0.01660 0.01675 0.01685 0.01685
      0.01715 0.01735 0.01745 0.01835
    ]

    frac = 0.6:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01525 0.01645 0.01650 0.01650 0.01650 0.01690 0.01690 0.01745
      0.01755 0.01780 0.01785 0.01805
    ]

    frac = 0.7:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01525 0.01560 0.01635 0.01640 0.01680 0.01680 0.01685 0.01700
      0.01700 0.01715 0.01775 0.01815
    ]

    frac = 0.8:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01580 0.01625 0.01625 0.01670 0.01680 0.01685 0.01705 0.01730
      0.01745 0.01750 0.01755 0.01855
    ]

    frac = 0.9:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01525 0.01595 0.01645 0.01650 0.01670 0.01685 0.01695 0.01710
      0.01725 0.01735 0.01740 0.01745
    ]

    frac = 1.0:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01570 0.01605 0.01620 0.01680 0.01680 0.01690 0.01715
      0.01720 0.01735 0.01770 0.01810
    ]

  There is some evidence here that mixing the two can improve cost at
  the end of construction, and that the improvement can carry through
  to final cost.  Here is an odd one that splits the two best results:

    frac = 0.65:
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01580 0.01605 0.01605 0.01640 0.01665 0.01665 0.01680 0.01685
      0.01690 0.01730 0.01745 0.01830
    ]

  But it's quite a lot worse than both, proving that there is a lot
  of randomness in these results.  More tests are needed.  The best
  result, 1525, has rel = 1.22.

  Added some randomness to dynamic.  It might be helpful when
  finding initial timetables for rigid resources.

14 May 2024.  Added a table of contents to HSEval's timetable and 
  report pages.  Returned to the frac = 0.6 result:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01630 0.01635 0.01640 0.01650 0.01650 0.01655 0.01690 0.01725
      0.01745 0.01770 0.01780 0.01790
    ]

  and had a look at the actual timetable.  Old KHE is from before,
  KHE is a full run now, and Cons is KHE at the end of construction:

                                     Leg   Old KHE   KHE  Cons  npg
    --------------------------------------------------------------
    U  Available times (positive)     24         25    27    39  50
    O  Available times (negative)     43         53    61    41  36
    Y  Unnecessary assignments        19         25    31    18   6
    X  Unassigned tasks                5          2     2    21  25
    --------------------------------------------------------------
    U - O + Y - X                     -5         -5    -5    -5  -5

  The npg column is KHE construction only, without profile grouping.
  It's hardly any different in U + Y.  Either way, KHE shows a
  blowout in U + Y.  Why did that happen?  Is optimal
  assignment choosing too many unnecessary assignments?  No, Cons
  shows that there are only 18 unnecessary assignments at the end
  of construction; but there are 21 unassigned tasks, and it is
  fixing these up that is making life hard for repair.  So the
  next job is to tweak construction so that there are fewer
  unassigned tasks at the end.

15 May 2024.  The questions are, would a matching really help?
  and if so what is the best way to add one?  Or would something
  quite different (e.g. stronger profile grouping) do better?
  Profile grouping interacts badly with optimal assignment.

  At the end of construction there seem to be a lot of resources
  whose workload limit is 20 and whose workload is 18.  Why did
  these not get a full workload?  The resources in question are
  HN_4, HN_14, HN_15, NU_21, NU_30, NU_31, CT_37, CT_38, CT_46,
  CT_52.  That's a lot of underloaded resources, but most of them
  are underloaded in Legrain as well.  It's because the only good
  way to get up to 20 is "busy 5 free 2" and repeat, but that
  pattern repeats every 7 days, and so it covers all four weekends
  or none.  (There is a maximum of 3 consecutive free days.)  Neither
  is a good option when the limit is 2.  To get closer to 2 busy
  weekends you have to give up 2 units of workload.

  I tried not installing cluster minimum limits but it made
  no difference.  Probably they are not biting anyway.

  TR_78 and TR_80 get high priority for optimal assignment.  Yet
  both use italic tasks.  Actually there isn't much else for them
  to do on that day.  But who says they had to work at all?  Am
  I giving those tasks the wrong weight?  Perhaps there is no
  penalty for using them, and there should be?  No - there is a
  penalty for not using the tasks that need assignment - that
  should do the trick.

16 May 2024.  Looking at TR_78 and TR_80.  There seems to be no
  reason why they could not be busy days other than 3Sat/3Sun, where
  non-italic tasks are available.  These should give better costs.

                  Busy    Weekends   C-Free C-Working
          Weight:  20         30        30     30
    -------------------------------------------------
    20Percent     3-6        0-1       2-7    2-5
    -------------------------------------------------

  Suitable pairs on 1Sat/1Sun are TR_88 and TR_96 only.  Suitable
  pairs on 2Sat/2Sun are TR_79 and TR_90.  Suitable pairs on
  4Sat/4Sun are TR_92 only.  But no-one says they have to work
  on weekends.  Only TR_75, TR_77, and TR_93 are overloaded,
  and only by 1 each.

  TR_80 could take two from TR_96 and move its whole timetable
  right by 3, which should give it access to various non-italic
  assignments.  Let's look in detail at why this did not happen.

  OK, I've worked it out.  TR_78 and TR_80 have history 5 in
  consecutive free days, so they have to start after 1 or 2
  free days.  This then forces their timetable to be as it is.

17 May 2024.  Worked out the second (history) measure of
  non-rigidity.  It's ready to implement.

18 May 2024.  Audited and implemented the second measure of
  non-rigidity.  All done and ready to test.

  This is what I was getting before m2:

    [ KheDynamicResourceSequentialSolve(INRC2-4-100-0-1108, Nurse:Trainee)
      frac 0.6, rt_count 25, rcount 15
      [ avail resources after sorting (there are 25):
	[TR_87 non_rigidity 0, avail 6, unavail 0, random 443, index 12]
	[TR_95 non_rigidity 0, avail 6, unavail 0, random 2919, index 20]
	[TR_86 non_rigidity 0, avail 6, unavail 0, random 5380, index 11]
	[TR_79 non_rigidity 0, avail 6, unavail 0, random 5430, index 4]
	[TR_94 non_rigidity 0, avail 6, unavail 0, random 7247, index 19]
	[TR_78 non_rigidity 0, avail 6, unavail 0, random 7455, index 3]
	[TR_80 non_rigidity 0, avail 6, unavail 0, random 7572, index 5]
	[TR_98 non_rigidity 0, avail 6, unavail 0, random 7974, index 23]
	[TR_96 non_rigidity 1, avail 20, unavail 0, random 5484, index 21]
	[TR_83 non_rigidity 1, avail 20, unavail 0, random 6138, index 8]
	[TR_88 non_rigidity 1, avail 20, unavail 0, random 7294, index 13]
	[TR_75 non_rigidity 1, avail 15, unavail 0, random 1120, index 0]
	[TR_93 non_rigidity 1, avail 15, unavail 0, random 3003, index 18]
	[TR_91 non_rigidity 1, avail 15, unavail 0, random 4230, index 16]
	[TR_90 non_rigidity 1, avail 15, unavail 0, random 6427, index 15]
	++++ drs above here, time sweep below here ++++
	[TR_92 non_rigidity 1, avail 15, unavail 0, random 6637, index 17]
	[TR_76 non_rigidity 1, avail 15, unavail 0, random 7522, index 1]
	[TR_82 non_rigidity 1, avail 15, unavail 0, random 9002, index 7]
	[TR_77 non_rigidity 2, avail 10, unavail 0, random 1298, index 2]
	[TR_89 non_rigidity 2, avail 10, unavail 0, random 1696, index 14]
	[TR_97 non_rigidity 2, avail 10, unavail 0, random 3110, index 22]
	[TR_99 non_rigidity 2, avail 10, unavail 0, random 4382, index 24]
	[TR_81 non_rigidity 2, avail 10, unavail 0, random 5926, index 6]
	[TR_85 non_rigidity 2, avail 10, unavail 0, random 7375, index 10]
	[TR_84 non_rigidity 2, avail 10, unavail 0, random 9115, index 9]
      ]
      ...
    ]

  And this is what I'm getting after m2:

    [ KheDynamicResourceSequentialSolve(INRC2-4-100-0-1108, Nurse:Trainee)
      frac 0.6, rt_count 25, rcount 15
      [ avail resources after sorting (there are 25):
	[TR_95 non_rigidity 3, avail 6, unavail 0, random 2919, index 20]
	[TR_86 non_rigidity 3, avail 6, unavail 0, random 5380, index 11]
	[TR_94 non_rigidity 3, avail 6, unavail 0, random 7247, index 19]
	[TR_78 non_rigidity 3, avail 6, unavail 0, random 7455, index 3]
	[TR_80 non_rigidity 3, avail 6, unavail 0, random 7572, index 5]
	[TR_79 non_rigidity 5, avail 6, unavail 0, random 5430, index 4]
	[TR_87 non_rigidity 6, avail 6, unavail 0, random 443, index 12]
	[TR_98 non_rigidity 7, avail 6, unavail 0, random 7974, index 23]
	[TR_96 non_rigidity 11, avail 20, unavail 0, random 5484, index 21]
	[TR_75 non_rigidity 11, avail 15, unavail 0, random 1120, index 0]
	[TR_93 non_rigidity 11, avail 15, unavail 0, random 3003, index 18]
	[TR_90 non_rigidity 11, avail 15, unavail 0, random 6427, index 15]
	[TR_76 non_rigidity 11, avail 15, unavail 0, random 7522, index 1]
	[TR_83 non_rigidity 12, avail 20, unavail 0, random 6138, index 8]
	[TR_91 non_rigidity 12, avail 15, unavail 0, random 4230, index 16]
	++++ drs above here, time sweep below here ++++
	[TR_82 non_rigidity 12, avail 15, unavail 0, random 9002, index 7]
	[TR_88 non_rigidity 13, avail 20, unavail 0, random 7294, index 13]
	[TR_92 non_rigidity 13, avail 15, unavail 0, random 6637, index 17]
	[TR_97 non_rigidity 21, avail 10, unavail 0, random 3110, index 22]
	[TR_89 non_rigidity 22, avail 10, unavail 0, random 1696, index 14]
	[TR_77 non_rigidity 23, avail 10, unavail 0, random 1298, index 2]
	[TR_81 non_rigidity 24, avail 10, unavail 0, random 5926, index 6]
	[TR_84 non_rigidity 24, avail 10, unavail 0, random 9115, index 9]
	[TR_99 non_rigidity 25, avail 10, unavail 0, random 4382, index 24]
	[TR_85 non_rigidity 25, avail 10, unavail 0, random 7375, index 10]
      ]
      ...
    ]

  And here are the statistics at the end of construction:

                                     Leg   Old Cons  Cons
    --------------------------------------------------------------
    U  Available times (positive)     24         39    43
    O  Available times (negative)     43         41    49
    Y  Unnecessary assignments        19         18    20
    X  Unassigned tasks                5         21    19
    --------------------------------------------------------------
    U - O + Y - X                     -5         -5    -5 

  Sadly they are slightly worse than before.  We've failed to remove
  the italic tasks from TR_78 and TR_80, basically because there are
  other resources that are just as rigid.  Here is a 5-minute 12-solve:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01630 0.01650 0.01660 0.01660 0.01665 0.01700 0.01720
      0.01725 0.01745 0.01835 0.01870
    ]

  It's not a bad result, rel = 1.24.  Probably best to simply slog
  on from here.

  Legrain's summary:

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (5 points) 	   	 150
    Avoid Unavailable Times Constraint (7 points)    	  70
    Cluster Busy Times Constraint (28 points) 	   	 950
    Limit Active Intervals Constraint (3 points)    	  75
    --------------------------------------------------------
      Grand total (43 points) 	   			1245

  KHE's summary (12 solves, 5 mins):

    Summary 					Inf. 	Obj.
    --------------------------------------------------------
    Assign Resource Constraint (3 points) 	   	  90
    Avoid Unavailable Times Constraint (7 points) 	  70
    Cluster Busy Times Constraint (34 points) 	   	1160
    Limit Active Intervals Constraint (11 points)    	 225
    --------------------------------------------------------
      Grand total (57 points) 	   			1545

  Supply and demand:
                                     Leg      KHE
    --------------------------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       55 
    Y  Unnecessary assignments        19       26 
    X  Unassigned tasks                5        3 
    --------------------------------------------------------------
    U - O + Y - X                     -5       -5

  This whole thing of mixing dynamic with time sweep has been
  worth doing, but we are still a long way short of the target,
  which is rel = 1.10 or cost = 1370.  If we could get rid of
  those 7 extra unnecessary assignments we would be getting there.
  Three of them are in the TR's, let's look into them.  I've looked
  into them and it's very hard to see how to get rid of any of them.

  Added a public KHE_MONITOR_ADJUSTER type, with the implementation
  in file khe_sm_monitor_adjustments.c.  Currently using it in vlsn,
  KheDetachLowCostMonitors, and KheSetClusterMonitorMultipliers.

  -------------------- Improving sequential -------------------------
  Trying to dream up ways to improve Sequential.  Can we do
  something more refined with the event resource monitors?
  To do it properly would require some kind of matching on
  each day.  Is that even feasible?  Could work it out once
  and for all in advance, set weights, then proceed as before.
  It's not the same thing as including global tixel matching
  monitors in the solve, because it's weighted.  But it might
  very well be supported by the resource matching solver,
  documented directly after optimal reassignment in the Guide.
  The key points seem to be

  (1) We can have one demand set per day, provided we can
      update it to fix the assignments of the resources
      that we've finished with.  As each resource is
      handled, its final timetable's assignments become
      fixed in the days.  The simple thing would be to
      start again for each resource on each day, but
      putting in a fix would probably be faster.

  (2) We don't want a general solution cost from these
      matchings (that would be meaningless), we just want
      the best possible total cost of event resource monitors
      for the one day.

  (3) We take the total cost of one matching as the assignment
      cost of assigning resource r to one task.  This will get
      incorporated into the solve in the usual way.  But it is not
      quite the constant that asst_cost is, because it varies as
      resources get fixed.  It is constant for each resource.  We
      need to store it in KHE_DRS_DAY, or perhaps KHE_DRS_TASK_ON_DAY,
      before each solve and retrieve it from there during the solve.

  Could we have an extra input to the dynamic solver, something like

     KheDynamicResourceSolverSetCostSupplement(KHE_RESOURCE r,
       KHE_TASK task, KHE_COST cost)

  The meaning would be to add cost to any solution in which r
  is assigned to task.  We could work out this cost outside
  the dynamic solver.  The interface is not quite right, it
  does not allow for r being free on a particular day.  And
  perhaps we need an mtask, not a task, although the cost will
  vary with the task.

  We probably can't check final cost against KHE, because of
  the extra costs contributed by the matchings.  But we can
  check that rerun cost equals original cost.  Probably best
  to just do it and then turn off any checks that fail.

  The problem is, I'm very doubtful about whether this would do
  better than what we are doing now.  At present we pay resources
  1 to assign themselves to a task with a hard assign resource
  constraint.  Under this scheme, they would be paid 0 until
  resources start to run out.
  -------------------- Improving sequential -------------------------

  ------------- matching during Sequential -------------------------
  I'm not sure that a matching would help the combined dynamic and
  time sweep initial construction.  Supposing it does, can we adapt
  the global tixel matching (unintegrated) to this new application?
  Some particular points:

  (1) We need to be sure that the matching is monitoring all tasks
      with a non-zero non-assignment cost.  I seem to have allowed
      for this in khe_sm_matching.c, where the matching solver has

        res->min_weight = KheCost(1, 0);  /* could be changed */

      This determines which tasks etc. are taken notice of, and
      it looks like it will do the job.

  (2) We need to be able to fix some assignments, perhaps just by
      carrying them out.  Yes, that should work; it's bread and
      butter for the global tixel matching.

  (3) We need to be able to say "this resource is not available for
      assignment on this day".  The straightforward way to do this
      would seem to be to add workload demand nodes to soak it all up.
      Do we even need to bother?  If we fill up a resource to its
      workload limit, there will be nothing left anyway.  Hmmm.
      There is the distinction between days and times that might
      wreck this idea.  Perhaps look into what the gtm is giving
      us now, or what it gives us with KheCost(0, 1).

  Is optimal assignment choosing too many unnecessary assignments?
  See discussion on 14 May 2024.  Perhaps we need that bipartite
  matching after all.  Unweighted perhaps, at each time.  Or
  if some selection reduces the maximum matching size, leave
  it out unless there is nothing else available.  But there is
  always a free day available - no, that can reduce the size too.
  We need to be told which resources are not assigned yet, they
  make up the supply nodes.  Perhaps a separate module that
  dynamic can call on to find out yes or no.  And the separate
  module can preserve the matching.  Could we use the regular
  tixel matching?  It is usually concerned with hard constraints
  only, we want non-zero constraints.  And we want it separate,
  we don't want integrated.  And we want to be able to say "this
  resource is not available for assignment on this day".

     bool Allow(task, r)
     {
       matching_cost_before = matching_cost(soln);
       if( !Assign(mtask, r) )
         return false;
       matching_cost_after = matching_cost(soln);
       Assign(mtask, r);
       return matching_cost_after <= matching_cost_before;
     }

  We need to be sure that the matching is monitoring all tasks
  with a non-zero non-assignment cost.
  ------------- matching during Sequential -------------------------

19 May 2024.  Fixed a bug in KheMonitorAdjusterChangeWeight, now
  running a test of various frac values as on 13 May but in a
  single run.  I should then go on and do it for all of INRC2-4.

  Deleted the "competitive solution sets only" table on the
  HTML Summary table of HSEval.  Fixed the running time table.

  I documented tilting the plateau, but then I began to think,
  and it occurred to me that limit active intervals monitors
  monitor the whole cycle, so that even though their defects
  could be anywhere in the cycle, the monitor as a whole can
  only be associated with the start of the cycle.  So that is
  a big obstacle to implementing an effective plateau tilt.
  I need to implement the tilt *inside* the monitor as well
  as outside it.  Can I do that?

  Started a long run to definitively test Sequential.

20 May 2024.  Here's the result of the long run, now saved
  in file long_frac_res.xml:

  Instances (20)         LOR17  24x12-0  24x12-2  24x12-4  24x12-6  24x12-8 24x12-10
  ----------------------------------------------------------------------------------
  INRC2-4-030-1-6291   0.01695  0.01835  0.01855  0.01885  0.01850  0.01825  0.01830
  INRC2-4-030-1-6753   0.01890  0.02005  0.01950  0.01985  0.01980  0.01995  0.01970
  INRC2-4-035-0-1718   0.01425  0.01630  0.01630  0.01635  0.01670  0.01685  0.01645
  INRC2-4-035-2-8875   0.01155  0.01270  0.01300  0.01330  0.01285  0.01335  0.01355
  INRC2-4-040-0-2061   0.01685  0.01900  0.01905  0.01930  0.01905  0.01865  0.01895
  INRC2-4-040-2-6106   0.01890  0.02090  0.02135  0.02135  0.02130  0.02135  0.02055
  INRC2-4-050-0-0487   0.01505  0.01660  0.01670  0.01640  0.01660  0.01690  0.01675
  INRC2-4-050-0-7272   0.01500  0.01660  0.01755  0.01700  0.01680  0.01680  0.01660
  INRC2-4-060-1-6115   0.02505  0.02840  0.02875  0.02860  0.02760  0.02775  0.02760
  INRC2-4-060-1-9638   0.02750  0.03115  0.03060  0.03080  0.02930  0.03025  0.03065
  INRC2-4-070-0-3651   0.02435  0.02845  0.02750  0.02740  0.02745  0.02710  0.02690
  INRC2-4-070-0-4967   0.02175  0.02600  0.02505  0.02610  0.02575  0.02435  0.02510
  INRC2-4-080-2-4333   0.03340  0.03705  0.03660  0.03595  0.03635  0.03680  0.03690
  INRC2-4-080-2-6048   0.03260  0.03825  0.03805  0.03760  0.03680  0.03630  0.03610
  INRC2-4-100-0-1108   0.01245  0.01630  0.01605  0.01685  0.01580  0.01585  0.01610
  INRC2-4-100-2-0646   0.01950  0.02285  0.02310  0.02305  0.02315  0.02430  0.02390
  INRC2-4-110-0-1428   0.02440  0.02705  0.02625  0.02585  0.02650  0.02695  0.02715
  INRC2-4-110-0-1935   0.02560  0.02940  0.02900  0.02895  0.02940  0.02930  0.02965
  INRC2-4-120-1-4626   0.02170  0.02500  0.02400  0.02465  0.02395  0.02460  0.02460
  INRC2-4-120-1-5698   0.02220  0.02635  0.02550  0.02620  0.02535  0.02590  0.02510
  ----------------------------------------------------------------------------------
  Average              0.02089  0.02383  0.02362  0.02372  0.02345  0.02357  0.02353

  With frac = 0.6 we save 2383 - 2345 = 38, presumably a reliable number
  since it is the average of so many components.  It's not a huge saving,
  only rel = 0.03, but it's something, and I can try it on other instances.
  Incidentally the averaget rel for frac = 0.6 is 2345 / 2089 = 1.12.  So
  the significance of the 0.03 is that 1.12 is much closer to the target
  than 1.15.  Halfway there, in fact.

  Implemented KheTiltPlateau.  It uses the monitor adjuster, which is
  all done, and also the tilt feature of limit active intervals monitors,
  which is also done and audited.  Testing now, here is a 5-minute run
  without KheTiltPlateau:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01610 0.01630 0.01670 0.01695 0.01700 0.01710 0.01725
      0.01725 0.01775 0.01835 0.01855
    ]

  And here is a 5-minute run with KheTiltPlateau enclosing the first
  call on ejection chains:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01570 0.01590 0.01620 0.01620 0.01635 0.01655 0.01670 0.01680
      0.01715 0.01750 0.01780 0.01840
    ]

  The best is worse, but (comparing pairwise) the rest are better.  And
  here we go with KheTiltPlateau enclosing both calls on ejection chains:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01555 0.01585 0.01600 0.01635 0.01650 0.01675 0.01715 0.01735
      0.01735 0.01735 0.01755 0.01795
    ]

  Again, the best is worse, but not much worse, and (comparing pairwise)
  many solutions do seem to be better.  There is enough here to justify
  another long run.  Not quite so long, because there are just two cases,
  tilt and no tilt.  All these tests were with frac=0.6.

  Here's a long comparison of regular (KHE24x12) and tilt (KHE24x12-T):

    Instances (20)           LOR17    KHE24x12  KHE24x12-T
    ------------------------------------------------------
    INRC2-4-030-1-6291     0.01695     0.01850     0.01810   *
    INRC2-4-030-1-6753     0.01890     0.01960     0.01970
    INRC2-4-035-0-1718     0.01425     0.01670     0.01610   *
    INRC2-4-035-2-8875     0.01155     0.01285     0.01310
    INRC2-4-040-0-2061     0.01685     0.01895     0.01850   *
    INRC2-4-040-2-6106     0.01890     0.02130     0.02085   *
    INRC2-4-050-0-0487     0.01505     0.01660     0.01610   *
    INRC2-4-050-0-7272     0.01500     0.01695     0.01655   *
    INRC2-4-060-1-6115     0.02505     0.02760     0.02745   *
    INRC2-4-060-1-9638     0.02750     0.02930     0.03045
    INRC2-4-070-0-3651     0.02435     0.02765     0.02745   *
    INRC2-4-070-0-4967     0.02175     0.02555     0.02580
    INRC2-4-080-2-4333     0.03340     0.03620     0.03645
    INRC2-4-080-2-6048     0.03260     0.03610     0.03715
    INRC2-4-100-0-1108     0.01245     0.01635     0.01595   *
    INRC2-4-100-2-0646     0.01950     0.02315     0.02380
    INRC2-4-110-0-1428     0.02440     0.02650     0.02645   *
    INRC2-4-110-0-1935     0.02560     0.02950     0.02935   *
    INRC2-4-120-1-4626     0.02170     0.02390     0.02445
    INRC2-4-120-1-5698     0.02220     0.02535     0.02625
    ------------------------------------------------------
        Average            0.02089     0.02343     0.02350

  Virtually indistinguishable, with tilt fractionally worse,
  although 11 of the 20 instances came out better (marked *).
  It's an open question whether tilt would do relatively
  better on a longer run time.

21 May 2024.  I've given some thought to whether there is anything
  that I could do to improve tilt, but I haven't been able to come
  up with anything.  So I'll revert to frac=0.6 without tilt and
  just slog on from there with INRC2-4-100-0-1108.

  -------------- Reference count in dynamic solutions --------------
  What about a reference count in dynamic solutions?  When a solution
  is created it increments its parent's reference count; when it is
  deleted (for example because it is dominated) it decrements its
  parent's reference count, and if the count falls to zero it
  deletes it, and so on recursively.  This would probably save a
  lot of memory, and consequently a lot of initialization time.
  But if the solution lies in a solution set, there will be an
  out of date reference from the solution set to the solution.
  We could include that reference in the count, and have an
  explicit deletion of each solution set after expanding each
  day.  Also we would need to take care not to delete a solution
  while it is being expanded.

  I've just had a look, and if I withdraw weak dominance then the
  only references to objects of type KHE_DRS_SOLN in objects are
  the array of them in soln lists and the prev_soln in soln objects.
  So keeping the reference counts in order should be pretty easy.
  But is it worth the trouble?  I'm not sure.

  A random fact about dynamic (that has become relevant because we
  are using it now to assign a single resource) is this:
  When there is a single resource, we can do optimal assignment even
  when there is grouping.  Just build the solution as many days ahead
  as the grouping takes us and add it to that day's solution set.
  -------------- Reference count in dynamic solutions --------------

  Here's the current state (including frac=0.6):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01650 0.01665 0.01675 0.01700 0.01705 0.01710 0.01710
      0.01735 0.01740 0.01770 0.01825
    ]

  Here rel = 1.29, no tilt.  The plan is to just slog on from here.

    Summary                                            Inf.   Obj.
    --------------------------------------------------------------
    Assign Resource Constraint (1 point)                        30
    Avoid Unavailable Times Constraint (8 points)               80
    Cluster Busy Times Constraint (34 points)                 1200
    Limit Active Intervals Constraint (15 points)              300
    --------------------------------------------------------------
      Grand total (58 points)                                 1610

                                     Leg      KHE
    --------------------------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       57 
    Y  Unnecessary assignments        19       25 
    X  Unassigned tasks                5        1 
    --------------------------------------------------------------
    U - O + Y - X                     -5       -5

  As usual it's the high number of unnecessary assignments that
  separates us most clearly from Legrain.  Although curiously we
  are doing better during the first two days of the cycle.  Let's
  look at all this again with tilt on:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01555 0.01570 0.01610 0.01610 0.01635 0.01645 0.01675 0.01680
      0.01690 0.01710 0.01770 0.01840
    ]

    Summary (KHE with tilt)                            Inf.   Obj.
    --------------------------------------------------------------
    Assign Resource Constraint (1 point)                        60
    Avoid Unavailable Times Constraint (8 points)              110
    Cluster Busy Times Constraint (34 points)                 1160
    Limit Active Intervals Constraint (15 points)              225
    --------------------------------------------------------------
      Grand total (58 points)                                 1555

                                     Leg      KHE (with tilt)
    --------------------------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       55 
    Y  Unnecessary assignments        19       25 
    X  Unassigned tasks                5        2 
    --------------------------------------------------------------
    U - O + Y - X                     -5       -5

  It's better but the difference is not striking.

  Started work on rs_drs_resources="same_shift(max_resources, days)".
  KheResourceSelectSameShiftReset and KheResourceSelectSameShiftChoose
  still to do, also working out how the resource select can influence
  the day select.

22 May 2024.  Got same_shift going, on the first test, a 5-minute run
  using same_shift(5, 5), it found this final solution:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01665 ]

  and same_shift ran 332 times and found two improvements.  So I tried
  same_shift(6, 6):

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01690 ]

  This time same_shift ran only 14 times and found no improvements.
  So let's go back to same_shift(5, 5) and try 12 5-minute solves:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01570 0.01610 0.01625 0.01660 0.01695 0.01700 0.01705 0.01710
      0.01710 0.01710 0.01755 0.01860
    ]

  Here rel = 1.26.  This is to be compared with old-style dynamic:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01650 0.01665 0.01675 0.01700 0.01705 0.01710 0.01710
      0.01735 0.01740 0.01770 0.01825
    ]

  We seem to have stumbled onto a genuine improvement here.  Here
  is a full run (12 5-minute solves each)

    Instances (20)           LOR17    KHE24x12  rel (by hand)
    ------------------------------------------
    INRC2-4-030-1-6291     0.01695     0.01840   1.08
    INRC2-4-030-1-6753     0.01890     0.01955   1.03
    INRC2-4-035-0-1718     0.01425     0.01680   1.17
    INRC2-4-035-2-8875     0.01155     0.01275   1.10
    INRC2-4-040-0-2061     0.01685     0.01840   1.09
    INRC2-4-040-2-6106     0.01890     0.02030   1.07  
    INRC2-4-050-0-0487     0.01505     0.01620   1.07
    INRC2-4-050-0-7272     0.01500     0.01645   1.09
    INRC2-4-060-1-6115     0.02505     0.02785   1.11
    INRC2-4-060-1-9638     0.02750     0.02930   1.06
    INRC2-4-070-0-3651     0.02435     0.02715   1.11
    INRC2-4-070-0-4967     0.02175     0.02560   1.17
    INRC2-4-080-2-4333     0.03340     0.03575   1.07
    INRC2-4-080-2-6048     0.03260     0.03605   1.10
    INRC2-4-100-0-1108     0.01245     0.01650   1.32
    INRC2-4-100-2-0646     0.01950     0.02305   1.18
    INRC2-4-110-0-1428     0.02440     0.02660   1.09
    INRC2-4-110-0-1935     0.02560     0.02940   1.14
    INRC2-4-120-1-4626     0.02170     0.02475   1.14
    INRC2-4-120-1-5698     0.02220     0.02525   1.13
    ------------------------------------------
        Average            0.02089     0.02330

  The average here of 2330 compares with the average using dynamic
  in the usual way (see 20 May) of 2343.  The difference is very
  small, (2343 - 2330) / 2089 = 0.006.  Notice that in this table
  the result for INRC2-4-100-0-1108 is 1650, whereas we got 1570
  before.  Must have been a lucky strike.  What about no dynamic?

    Instances (20)           LOR17    KHE24x12
    ------------------------------------------
    INRC2-4-030-1-6291     0.01695     0.01850
    INRC2-4-030-1-6753     0.01890     0.01970
    INRC2-4-035-0-1718     0.01425     0.01665
    INRC2-4-035-2-8875     0.01155     0.01270
    INRC2-4-040-0-2061     0.01685     0.01825
    INRC2-4-040-2-6106     0.01890     0.02045
    INRC2-4-050-0-0487     0.01505     0.01610
    INRC2-4-050-0-7272     0.01500     0.01645
    INRC2-4-060-1-6115     0.02505     0.02760
    INRC2-4-060-1-9638     0.02750     0.02930
    INRC2-4-070-0-3651     0.02435     0.02745
    INRC2-4-070-0-4967     0.02175     0.02555
    INRC2-4-080-2-4333     0.03340     0.03575
    INRC2-4-080-2-6048     0.03260     0.03685
    INRC2-4-100-0-1108     0.01245     0.01530
    INRC2-4-100-2-0646     0.01950     0.02360
    INRC2-4-110-0-1428     0.02440     0.02560
    INRC2-4-110-0-1935     0.02560     0.02915
    INRC2-4-120-1-4626     0.02170     0.02395
    INRC2-4-120-1-5698     0.02220     0.02555
    ------------------------------------------
        Average            0.02089     0.02322

  So no dynamic at all is marginally better.  But I showed
  previously that dynamic finds optimal solutions to several
  of the COI instances.  Notice here the good result for
  INRC2-4-100-0-1108 of 1530, which is rel = 1.22.  And
  the average rel = 1.11, which is not far off good enough.
  The result for INRC2-4-030-1-6291 is rel = 1.09.

  Back to the current standard (default values plus frac = 0.6):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01660 0.01665 0.01685 0.01690 0.01695 0.01710 0.01720
      0.01760 0.01775 0.01835 0.01870
    ]

  We need to move on from here.  We have rel = 1.29, not great.

23 May 2024.  Working on focussing same_shift onto limit active
  intervals defects.  But first I've embarked on a pretty large
  rejig of the module.  So far I've removed solve_args from runner,
  changed runner to VLSN Solver, added KHE_RESOURCE_SELECT_SINGLE,
  broken KHE_DAY_SELECT into a hierarchy with three concrete subtypes,
  and got rid of boilerplate parameters by passing a VLSN solver.

24 May 2024.  Did a careful audit of vlsn.  It's pretty good now,
  I tidied up a few things.

25 May 2024.  I seem to have finished defective_shift, although it
  all needs a good audit.

26 May 2024.  Finished auditing the new code, started testing.  Here
  is one example where the new code found an improvement:

    [ KheDynamicResourceSolverDoSolve(drs, priqueue false, extra_selection true,
      expand_by_shifts true, shift_pairs false, correlated_exprs false,
      daily_expand_limit 10000, daily_prune_trigger 0,
      resource_expand_limit 0, dom_approx 0,
      main_dom_kind IndexedTabulated, cache false, cache_dk -) cost 0.01680
      resources:  TR_82, TR_86, TR_89, TR_96
      day ranges: 5-9 (1Sat{1Sat1..1Sat4} - 2Wed{2Wed1..2Wed4})
    ] KheDynamicResourceSolverDoSolve ret. true (new 0.01665 < old 0.01680)

  This is the only example of improvement on the first test run.  Here
  is a 5-minute 12-solve with rs_drs_resources="defective_shift(5,5)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01625 0.01675 0.01685 0.01700 0.01705 0.01710 0.01750
      0.01750 0.01755 0.01765 0.01775
    ]

  The best solution is the same but the worst solution is much better.
  Here's another run, ostensibly with the same parameters:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01605 0.01655 0.01660 0.01670 0.01685 0.01685 0.01685
      0.01715 0.01715 0.01730 0.01820
    ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01605 0.01655 0.01660 0.01670 0.01685 0.01685 0.01700
      0.01700 0.01715 0.01750 0.01870
    ]

  Not sure why it turned out better.  Rel = 1.24.  If we change to
  defective_shift(6,6) we get this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01655 0.01665 0.01685 0.01690 0.01695 0.01695 0.01710
      0.01710 0.01720 0.01725 0.01740
    ]

  Not quite so good.

27 May 2024.  Working on the documentation of targeted VLSN search.
  This is an idea with a lot of potential.  Documentation is done
  except that assign resource, prefer resources, and limit resources
  defects are not yet documented.

28 May 2024.  Carrying on with the implementation of targeted VLSN
  search.  Targeted resource selects all done, but now I need to
  implement each redefinition of KheVLSNMonitorRandom.  I've done
  it for KheVLSNLimitActiveIntervalsMonitorRandom, and audited it.

29 May 2024.  Working on targeted VLSN search.  Documented and
  implemented the definitive way to grow or shrink a sequence of
  days D.  Wrote descriptions for assign resource, prefer resources,
  and limit resources monitors.

  Wrote KheVLSNAvoidUnavailableTimesMonitorRandom.  Then I did a
  pretty good job of auditing it and making it and the doc agree.

  Altered KheResourceTimetableMonitorTimeGroupAvailable so that it
  checks whether any of the times are unavailable, by hunting
  through the monitored time groups at each time.  This will be
  a lot faster than anything else I can do easily.

31 May 2024.  Finished revising my PATAT 2024 paper in response to
  referees' reports.  All done and ready to submit.

1 June 2024.  Revised the description of how VLSN neighbourhoods
  are built for each type of defect.  All good now.

  Avoid unavailable times says "and not subject to any avoid unavailable
  times monitors at @M { t }".  I've now implemented this, including
  changing the spec of KheResourceTimetableMonitorTimeGroupAvailable.

  KheVLSNClusterBusyTimesMonitorRandom, KheVLSNLimitBusyTimesMonitorRandom,
  and KheVLSNLimitWorkloadMonitorRandom all written.  That finishes off
  the resource monitors.  Just done assign resource, and audited it.

2 June 2024.  Did some rewriting of the general blurb that comes before
  the descriptions of the neighbourhoods.  The main new point is that
  for coherence we should choose sets of resources R which are similar.
  So now I need to review the neighbourhood descriptions to make this
  happen.  It will explain what to do with a prefer resources monitor
  whose domain is empty.

3 June 2024.  Rewrote the targeted VLSN search doc - again.  It's
  good although limit active intervals defects are still a work in
  progress, and I need to work the same-shift thing into it somehow.

4 June 2024.  Finished the rewrite of the targeted VLSN search doc.
  I've audited it and verified that it really is ready to implement now.

  Did a thorough revamp of the availability functions in resource
  timetable monitors.  Much clearer now.  Am using the new functions
  throughout KHE and have a clean compile.

  Rewritten KheVLSNAssignResourceMonitorRandom and
  KheVLSNPreferResourcesMonitorRandom.  

5 June 2024.  More slogging away at the selection functions.  I've
  finished up to the end of avoid unavailable times.  All implemented
  and audited.

6 June 2024.  Working on KheVLSNClusterBusyTimesMonitorRandom.  All
  done, except needs an audit, and a lot of useful code was written
  along the way.

7 June 2024.  Finished off KheVLSNClusterBusyTimesMonitorRandom and
  KheVLSNLimitBusyTimesMonitorRandom.  All documented, implemented,
  and audited, including the allow_zero case.

8 June 2024.  Finished KheVLSNLimitActiveIntervalsMonitorRandom.
  It needs an audit.

  Finished revising my PATAT 2024 paper and submitted.  They also
  asked for a .zip or .tgz of the source so I submitted a .zip.

9 June 2024.  Rewrote the documentation so that it now presents
  "separate" and "combined" values for the rs_drs_resources and
  rs_drs_days options, now coalesced into just rs_drs.  I've
  updated the type definitions suitably and now I am working
  my way through the submodules, just updating everything that
  needs it as I go.

10 June 2024.  Reorganizing khe_sr_dynamic_vlsn.c.  Done everything
  down to a clean compile, audit, and test.  The test seems to be
  working well, although it hasn't found any improvements yet.  The
  only serious bug was that my treatment of limit active intervals
  monitors was simple-minded.  I had to review what I'd done
  previously in khe_se_solvers.c and redo it that way.

11 June 2024.  Off-site backup done today.  Did some more testing
  of INRC2-4-100-0-1108, getting around 1675 or 1700.  No sign of
  any miraculous improvement.  Here is a 5-minute 12-solve with
  rs_drs="defective_shift(5,5)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01600 0.01645 0.01670 0.01680 0.01690 0.01695 0.01710 0.01710
      0.01725 0.01730 0.01750 0.01750
    ]

  This is typical of what I was getting before.  Now the same run
  again but with rs_drs="targeted(1:112, 2:56, 3:28, 4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01610 0.01655 0.01670 0.01690 0.01710 0.01710 0.01720
      0.01735 0.01740 0.01780 0.01780
    ]

  1545 is rel 1545 / 1245 = 1.24.  I've been getting this 1545
  previously too.  Now here is targeted again but omitting cluster
  monitors with more than 4 time groups:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01535 0.01660 0.01685 0.01685 0.01700 0.01700 0.01705 0.01715
      0.01720 0.01720 0.01740 0.01790
    ]

  The best here is a new best for me, rel = 1.23.  But look at the
  second best.  I need to slow down and improve the targeting, I
  suspect.  But for now I will continue with omitting large cluster
  defects.  Here is rs_drs="targeted(3:28, 4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01675 0.01685 0.01685 0.01695 0.01710 0.01710 0.01710
      0.01725 0.01745 0.01770 0.01805
    ]

  I've changed the neighbourhood shape code so that it always
  chooses a random shape from the full list.  It's up to the
  user now to not include entries for 1 and 2 resources.  And
  here is rs_drs="targeted(4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01535 0.01610 0.01660 0.01680 0.01700 0.01705 0.01705 0.01720
      0.01720 0.01720 0.01775 0.01780
    ]

  This is what we got before.  Now allowing all cluster monitors:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01630 0.01675 0.01690 0.01695 0.01700 0.01710 0.01710
      0.01735 0.01735 0.01740 0.01780
    ]

  The difference is basically random.  But we do need to improve
  the targeting of targeted VLSN as much as we can.  Here is another
  run, with rs_drs="targeted(5:6)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01635 0.01665 0.01670 0.01685 0.01710 0.01710 0.01715
      0.01715 0.01735 0.01805 0.01890
    ]

  This is quite poor, so we'll can it now.  rs_drs="targeted(4:8, 5:6)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01595 0.01610 0.01640 0.01675 0.01685 0.01710 0.01710 0.01725
      0.01735 0.01740 0.01780 0.01900
    ]

  And finally back to rs_drs="targeted(4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01660 0.01660 0.01675 0.01695 0.01705 0.01720 0.01730
      0.01735 0.01740 0.01765 0.01775
    ]

  And sticking with rs_drs="targeted(4:8, 5:6, 6:3)" but omitting
  long cluster monitors:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01605 0.01650 0.01650 0.01660 0.01660 0.01690 0.01705 0.01720
      0.01720 0.01750 0.01790 0.01875
    ]

  Now back to omitting long cluster monitors but with
  rs_drs="targeted(3:28, 4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01610 0.01650 0.01665 0.01685 0.01695 0.01695 0.01710 0.01725
      0.01725 0.01730 0.01730 0.01820
    ]

  It's probably time now to look at the defects and think of ways
  to grind them down.  Targeted VLSN search is a logical thing to
  try, even if it has not changed the world, so let's take it on
  and just keep going.

  The differences are largely random but I think on the whole I
  should persist with rs_drs="targeted(4:8, 5:6, 6:3)" and not
  omitting large cluster monitors.  Here we are doing that for a
  60-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01450 0.01510 0.01555 0.01575 0.01575 0.01580 0.01580 0.01585
      0.01615 0.01615 0.01615 0.01640
    ]

  This 60-minute run has produced rel = 1.16, which has got to be
  my best result so far, although still a long way from where we
  want to be.  I can't really compare it with earlier runs, there
  don't seem to be any 60-minute INRC2-4-100-0-1108 runs.  Here is
  another 60-minute run, with rs_drs="targeted(3:28, 4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01450 0.01510 0.01555 0.01575 0.01580 0.01580 0.01585 0.01595
      0.01600 0.01615 0.01615 0.01640
    ]

  There's nothing between this result and the previous one.  And
  here is another 60-minute run, limiting to small cluster monitors:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01475 0.01510 0.01540 0.01565 0.01570 0.01575 0.01580 0.01585
      0.01590 0.01605 0.01615 0.01640
    ]

  There are differences here but they are essentially random.
  So I've decided to not limit to small cluster monitors, at
  least for the time being.  Just for fun, here is a 2-hour run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 120.0 mins:
      0.01450 0.01510 0.01555 0.01575 0.01580 0.01580 0.01580 0.01585
      0.01585 0.01615 0.01615 0.01640
    ]

  As it says, it really did run for 2 hours; but it produced nothing
  better.  Food for thought there.

12 June 2024.  Done a few five-minute runs to see what't happening.
  Actually vlsn is getting practically no time to shine.  It seems
  that ejection chains is soaking it all up, so I'll try 50-50.
  I'm currently getting solution cost 1690 from the 9:1 5-minute run.
  And from the 5:5 run 1720, including one DRS success.  Some of
  the individual DRS runs seemed a bit slow, so now I'm trying

    targeted(4:7, 5:5, 6:3)

  We are getting a much larger number of calls, and some are
  successes, so it feels better.  But the final cost was 1730.

13 June 2024.  Documented a tightening up of the choice of
  resources.  Started implementing it.  So far I have written
  KheVLSNSolverResources, it does the job but now needs to be
  called by the neighbourhoods.

14 June 2024.  Added KHE_PREFER type hierarchy, and am now
  using it.  All done to a clean compile and audited.

15 June 2024.  Audited everything at or below KHE_SELECT_TARGETED.
  All good and ready to test.  I started work on targeted VLSN
  search on 27 May 2024.  So almost three weeks ago - amazing.
  Best of one, with rs_drs="targeted(4:7, 5:5, 6:3)":

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01695 ]

  There were 7 successes:

    success: 0.02690 -> 0.02680
    success: 0.02680 -> 0.02670
    success: 0.02670 -> 0.02660
    success: 0.02660 -> 0.02650
    success: 0.01780 -> 0.01770
    success: 0.01770 -> 0.01760
    success: 0.01710 -> 0.01695

  including the one that got us down to 1695.  Best of 12:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01705 0.01720 0.01725 0.01735 0.01745 0.01760 0.01765 0.01770
      0.01835 0.01885 0.01905 0.01970
    ]

  The total number of successes is 68, or 68/12 = 6 per solve.

  I've looked over the sets of resources and they are much more
  coherent now - possibly too coherent.  I could change 2*count
  to 3*count quite easily now to decrease the coherence.

  Trying rs_drs="targeted(4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01690 0.01720 0.01720 0.01740 0.01815 0.01815 0.01840 0.01840
      0.01875 0.01890 0.01910 0.01955
    ]

  The total number of successes is 28, or 28/12 = 2 per solve.
  Not many successes, but on balance the result is better.

  Replacing 2*(count - 1) by 3*(count - 1).  The sets of resources
  still look very coherent.  Anyway the result is

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01670 0.01685 0.01740 0.01765 0.01770 0.01805 0.01845 0.01850
      0.01885 0.01925 0.01980 0.01985
    ]

  with 25 successes.  Now trying 4*(count - 1):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01650 0.01715 0.01740 0.01745 0.01745 0.01750 0.01805 0.01875
      0.01885 0.01915 0.01970 0.01990
    ]

  with 21 successes.  Now trying 5*(count - 1):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01680 0.01695 0.01780 0.01815 0.01815 0.01820 0.01820 0.01825
      0.01835 0.01915 0.01930 0.01975
    ]

  Now back to 3*(count - 1), but with rdv only during the second
  repair phase, making it truly a method of last resort:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01615 0.01635 0.01685 0.01690 0.01715 0.01725 0.01755 0.01775
      0.01775 0.01785 0.01795 0.01805
    ]

  This is quite a lot better.  There were only 5 successes:

    success: 0.01830 -> 0.01825
    success: 0.01915 -> 0.01905
    success: 0.01845 -> 0.01835
    success: 0.01835 -> 0.01815
    success: 0.01770 -> 0.01755

  These don't look like last resort improvements.  Trying
  again with rs_drs="targeted(3:28, 4:8, 5:6, 6:3)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01590 0.01615 0.01655 0.01675 0.01710 0.01715 0.01725 0.01730
      0.01745 0.01800 0.01835 0.01980
    ]

  This seems better, but there were no successes:  everything here
  was achieved by ejection chains.  Rel = 1590 / 1245 = 1.27.  Here
  is rs_drs="same_shift(5, 5)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01615 0.01655 0.01670 0.01680 0.01715 0.01730 0.01755 0.01755
      0.01755 0.01770 0.01815 0.01835
    ]

  and here is rs_drs="defective_shift(5, 5)":

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01615 0.01675 0.01685 0.01710 0.01715 0.01745 0.01770
      0.01785 0.01795 0.01815 0.01935
    ]

  There were 18 successes but they don't look like last resort
  successes.  The best was 1625.  Rel = 1.25.  We are still a
  long way above where we want to be.  Why?

16 June 2024.  Decided to focus on rs_drs="targeted(4:8, 5:6, 6:3)"
  with rs_drs_seq_frac=0.6 and go back to poring over the defects.

  Trying fixing the initial optimal assignments.  Here is a first run:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.02380 ]

  It seems to be working, but I would have to adjust quite a
  lot of other code so that it doesn't waste time fiddling with
  fixed resources.  I've decided against pursuing it now.  I did
  notice some optimally assigned resources that were shirking
  their fair share of weekend work.

17 June 2024.  Doing the preparatory reading for updating profile
  grouping, especially as it relates to history.  There is a lot of
  code:  khe_sr_tasker.c, khe_sr_comb_solver.c, khe_sr_group_by_rc.c.
  The documentation is not wonderfully intelligible.

18 June 2024.  Working on a new "Profile grouping and history"
  section of the Guide.  I'm up to the business end of it now.

19 June 2024.  Carrying on with khe_sr_group_by_history.c.  I've
  done quite a lot of boilerplate, gathering constraints, resources,
  and history values.

20 June 2024.  Carrying on with khe_sr_group_by_history.c.  Working
  now to build the matching graphs using mmatch.h.

23 June 2024.  Not sure where the last two days went.  Still carrying
  on with khe_sr_group_by_history.c.

24 June 2024.  Still carrying on with khe_sr_group_by_history.c.
  I'm rewriting the doc, rather well, to explain exactly what
  gets matched with what.

26 June 2024.  Carrying on rewriting the group by history doc.

28 June 2024.  Finished rewriting the group by history doc and
  began work on the re-implementation.

29 June 2024.  Audited KheConstraintClassBuildTaskNodes again,
  it's all good.

30 June 2024.  Redone the documentation including a beautifully
  simple pseudo-code presentation of the main algorithm.  Now I
  need to bring the implementation into line with that.

1 July 2024.  Done a final audit of doc and code.  All good, but
  there are a few points at the end of the doc that need looking into.

2 July 2024.  Working on assign cost and non-assign cost.  So far
  I've done the boilerplate of getting these values into task
  nodes, including keeping them up to date as nodes are merged.

3 July 2024.  Lost most of today to silly jobs.  Worked on the
  revised doc, and got to a point where I have to work out how
  to combine the two parts of edge cost in the ymatch.

4 July 2024.  I've decided to go back to analysis and work out what I
  want to achieve with grouping by history.  Here is a 5-minute run with
  rs_drs="targeted(4:8, 5:6, 6:3)" rs_drs_seq_frac=0.6 rs_drs_fix=true:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.02380 ]

  And here it is again with rs_drs_fix=false:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01770 ]

  I forget what the point of rs_drs_fix was.  So now here is a best
  of 12 run with rs_drs_fix=false:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01675 0.01705 0.01730 0.01735 0.01740 0.01790 0.01795 0.01815
      0.01820 0.01855 0.01890 0.01915
    ]

  Here rel = 1675 / 1245 = 1.34.  A long way to go.

  I'm paying 120 for history problems, LOR is paying 75.  It should
  be easy to do as well as LOR is doing, you would think.  And a
  saving of 55 would be 0.04 off rel.  But does this really deserve
  the priority that I have been giving it?

  Here is best of 12 with rs_drs_fix=true:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.02130 0.02275 0.02295 0.02305 0.02310 0.02335 0.02395 0.02395
      0.02405 0.02405 0.02430 0.02435
    ]

  Back to rs_drs_fix=false, but now with rs_group_by_rc_off=true:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01700 0.01710 0.01725 0.01735 0.01770 0.01785 0.01815 0.01865
      0.01900 0.01930 0.01980 0.02050
    ]

  It's inferior to running with grouping.  But what grouping are
  we getting?  Here we are with rs_drs_seq_frac=0.0:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01650 0.01725 0.01725 0.01755 0.01765 0.01775 0.01820 0.01825
      0.01835 0.01860 0.01885 0.01905
    ]

  This is about the same as with rs_drs_seq_frac=0.6.  Rel = 1.32.

5 July 2024.  Let's look at our old friend, U - O + Y - X:

                                     Leg      KHE
    --------------------------------------------------------------
    U  Available times (positive)     24       30
    O  Available times (negative)     43       60 
    Y  Unnecessary assignments        19       26 
    X  Unassigned tasks                5        1 
    --------------------------------------------------------------
    U - O + Y - X                     -5       -5

  The difference in U + Y is 6 + 7 = 13, at 20 points each that
  comes to 260, which has rel = 0.2, which is massive.

  Comparing the KHE solution to LOR, KHE has 17 overloads among the
  HN nurses, LOR has only 9.  KHE has only 2 more available times
  among the HN nurses than LOR, so KHE must be renting out more
  HN nurses for ordinary duries.

  Concerning limit active intervals, the points differences are:

                               LOR     KHE
     ------------------------------------------
     Involving history          75     120
     Not involving history       0     150
     ------------------------------------------
       Total                    75     270

  So actually I'm further behind on the non-history cases than on the
  history cases.  And 150 is a serious amount:  rel = 0.12.

  Looked at what grouping I was getting.  There seems to be a lot of
  it.  Some is quite predictive of LOR's solution, others not so much.
  e.g. in trainees' night shifts the grouping seems to predict LOR
  quite well in the earlier weeks, but not in Week 4.

  Did some minor adjustments to the default value of rs and got this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01645 0.01665 0.01675 0.01685 0.01710 0.01725 0.01735 0.01735
      0.01775 0.01810 0.01845 0.01880
    ]

  It's better throughout, let's leave it as is.  It might be because
  it gives more time to ejection chains and less to dynamic, but anyway.
  With rs_drs_seq_frac=0.6 we get this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01530 0.01655 0.01665 0.01675 0.01675 0.01695 0.01730 0.01750
      0.01750 0.01755 0.01780 0.01860
    ]

  It's basically an outlier, but what an outlier:  rel = 1.22.

  Counting augments to see how much difference it makes running in
  parallel.  On a parallel run with 12 threads each thread averaged
  4924.83 augments.  On a single-threaded run I got 6085.  So that
  suggests a slowdown of 0.8.

  In the outlier 1530 solution, the limit active intervals numbers are

                               LOR     KHE
     ------------------------------------------
     Involving history          75     120
     Not involving history       0      90
     ------------------------------------------
       Total                    75     210

  Virtually all of the cost (all but 45) comes from Constraint
  17, which is night shifts, and always it's failing the min
  limit of 4 consecutive night shifts.  And we have

                                     LOR      KHE
    --------------------------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       56 
    Y  Unnecessary assignments        19       25 
    X  Unassigned tasks                5        2 
    --------------------------------------------------------------
    U - O + Y - X                     -5       -5

  LOR is 10 ahead, which at cost 20 each is 200, which is rel = 0.16
  which is massive.  So U + Y is definitely the problem here.

  Here is rs_drs_seq_frac=0.6 with tilt on.  Hmmm, that did not
  work, it looks like tilt does not work with dynamic.  So here
  is a version that applies tilt only to ejection chains:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 294.8 secs:
      0.01600 0.01610 0.01620 0.01655 0.01660 0.01680 0.01695 0.01765
      0.01780 0.01780 0.01805 0.01805
    ]

  Not really much different than without tilting, and we've lost
  the outlier.

6 July 2024.  If we made these changes:

   Swap CT_44 CT_47 2Mon-2Fri
   Swap CT_47 CT_54 2Fri

  we would probably fix the CT_54 3 nights defect.  But it's
  hard to understand this repair within the current ejection
  chains framework.  Something like this:

     "When moving a task to remove an underload, we prefer
     to move it from an overloaded resource.  If there is
     no suitable overloaded resource, try to swap a whole
     sequence so that the desired task is first moved to
     an overloaded resoure, then it can be moved from there."

  So it's a more elaborate repair, not a chain as such.
  A single move won't work because the laim defect has
  cost 15 and a single moce will cost 20 in workload.

  What about this:  "Periodically make zero-cost moves
  of equal-length sequences of busy days at random".
  Much more hit and miss though.

    "When moving a task to correct an underload, if at
    level 1, first check whether the task is coming from
    one end of a sequence of consecutive busy days for
    an overloaded resource.  If not, try to find a zero
    cost swap of that task's sequence of busy days with
    a similar sequence of busy days in some other resource
    which is overloaded.  Do that swap plus the original swap."

  Reviewed the unassign_extreme_unneeded parameter of
  KheMTaskSetResourceReassign.  Here is a run where it has
  been turned off:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 294.8 secs:
      0.01690 0.01695 0.01700 0.01725 0.01745 0.01750 0.01755 0.01785
      0.01795 0.01855 0.01925 0.01955
    ]

  And here it is turned on again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01645 0.01655 0.01660 0.01675 0.01685 0.01700 0.01710 0.01740
      0.01750 0.01760 0.01770 0.01875
    ]

  It's making a real difference.  It was not documented, but it
  is documented now.

  When increasing workload in time group tg, I'm excluding other
  resources now that are now busy during tg.  Here's a run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 298.8 secs:
      0.01580 0.01655 0.01675 0.01700 0.01745 0.01755 0.01755 0.01760
      0.01770 0.01795 0.01820 0.01895
    ]

  The best score is better but 10 of the 12 are worse.

6 July 2024.  Doing double swaps.  KheIncreaseLoadDoubleSwapMultiRepair
  and KheDoubleSwapRepair are done.  Testing them now.  First results:

    [ "INRC2-4-100-0-1108", 1 solution, in 282.2 secs: cost 0.01620 ]

  And best of 12:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 297.9 secs:
      0.01560 0.01590 0.01700 0.01725 0.01725 0.01735 0.01740 0.01755
      0.01765 0.01770 0.01775 0.01795
    ]

  This seems to be better on the whole; rel = 1.25.  I've looked at
  the detailed list of defects.  I have only four history-related
  defects, compared to LOR which has three history-related defects.
  Looks like I don't need grouping by history.  Here's U - O + Y - X:

                                     LOR      KHE (prev)  KHE (now)
    --------------------------------------------------------------
    U  Available times (positive)     24       28          28
    O  Available times (negative)     43       56          56
    Y  Unnecessary assignments        19       25          25
    X  Unassigned tasks                5        2           2
    --------------------------------------------------------------
    U - O + Y - X                     -5       -5          -5

  It hasn't changed, the improvement must be in limit actives.
  Here's a 15-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 14.0 mins:
      0.01460 0.01540 0.01570 0.01570 0.01605 0.01615 0.01615 0.01625
      0.01630 0.01630 0.01675 0.01720
    ]

  That's a good result, rel = 1.17.  Almost as good as the 1450 I
  got from a 2-hour run on 11 June 2024.  Here's a two-hour run now:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 88.8 mins:
      0.01460 0.01530 0.01545 0.01570 0.01600 0.01605 0.01615 0.01615
      0.01620 0.01630 0.01635 0.01725
    ]

  No better than the 15-minute run.  Interesting.

9 July 2024.  Yesterday I documented global load balancing.  Today
  I've audited the documentation and started to implement, in
  file khe_sr_global_load.c.  Also added it to yourself but not
  documented it in yourself section yet.

10 July 2024.  Implementing global load balancing.  Have done some
  testing, the code is currently creating resources and their task
  sets of a given length.

11 July 2024.  Now building the graph, except not calculating the
  change in cost yet.  Doing that is next.

12 July 2024.  Still working on global load balancing.  I seem to
  have implemented it in a way that is different from, and better
  than, the documentation, so today I rewrote the documentation.  Also
  updated the implementation.  All done except KheGlbGraphSolve.
  Did some debug printing, I have not built the graph correctly.

13 July 2024.  Building the graph right today.  All written and
  ready to test.  Seems to be working.  I've also written a
  breadth-first search and am using it to pare down the graph
  to nodes that lie on a path from a source to a sink.  It all
  seems to be working (I have debug output showing the breadth
  first search reaching lots of nodes) but the search never
  reaches a sink node.

  Just to test, I've moved the call to earlier in the solve.
  And indeed I am getting some paths from source to sink now.
  Not very many.

14 July 2024.  Got the whole thing working beautifully now.
  About a week since I started work on it (before 9 July).
  The documentation is up to date too.  I'm not bothering to
  check whether the same resource is visited twice, it doesn't
  seem to be happening.  If the path does not produce a cost
  reduction, we skip it.  Also added time limit checks.

15 July 2024.  Should be able to build the graph while doing
  a breadth-first search of it.  Got that working today, after
  fixing one silly bug.  It seems to be running much faster,
  6 seconds rather than 110, while finding the same paths.

  Then I merged all the lengths into a single graph.  It
  seems to have worked first time, but it does take longer
  to run:  22 secs for Nurse, 0.2 secs for trainees.

  Here is a full run with rgl on:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 292.6 secs:
      0.01605 0.01625 0.01700 0.01705 0.01720 0.01735 0.01740 0.01740
      0.01770 0.01805 0.01805 0.01840
    ]

  And here with rgl off:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 299.8 secs:
      0.01610 0.01700 0.01710 0.01725 0.01735 0.01755 0.01755 0.01765
      0.01765 0.01790 0.01790 0.01810
    ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 299.7 secs:
      0.01580 0.01590 0.01670 0.01695 0.01700 0.01735 0.01740 0.01775
      0.01780 0.01795 0.01840 0.01845
    ]

  The differences are probably random, although interesting.  I've
  decided to leave rgl off for now.  Cost 1610 is rel = 1.29, 1580
  is 1.26.  Still a long way off the pace.  Here's a 15-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 14.2 mins:
      0.01460 0.01555 0.01570 0.01580 0.01605 0.01610 0.01615 0.01615
      0.01625 0.01650 0.01675 0.01735
    ]

  This is rel = 1.17, as we saw before.  Here is a 15-minute run with rgl:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 12.7 mins:
      0.01505 0.01560 0.01570 0.01580 0.01600 0.01605 0.01615 0.01615
      0.01630 0.01660 0.01680 0.01730
    ]

  Actually this had rgl after rec.  If we put it before:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 13.8 mins:
      0.01460 0.01535 0.01555 0.01570 0.01605 0.01610 0.01615 0.01615
      0.01625 0.01650 0.01685 0.01730
    ]

  This is basically no different.  We're leaving rgl out now.

16 July 2024.  Trying yesterday's 15-minute run with a more random
  choice of resources for sequential:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 14.0 mins:
      0.01475 0.01535 0.01545 0.01610 0.01620 0.01630 0.01655 0.01665
      0.01670 0.01695 0.01720 0.01765
    ]

  No significant difference, or possibly worse; let's cancel this.

  Returning now to 5-minute runs.  I've changed the interval iterator
  so that it only iterates in one direction when repairing limit
  active intervals defects that are too short.  I get this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.9 secs:
      0.01540 0.01585 0.01625 0.01630 0.01660 0.01715 0.01720 0.01720
      0.01725 0.01830 0.01835 0.01895
    ]

  For a five-minute run this is a good result.  Can I make further
  time savings?  This got by limiting the direction when repairing
  limit active intervals defects that are too long:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.7 secs:
      0.01585 0.01600 0.01600 0.01615 0.01660 0.01700 0.01725 0.01750
      0.01760 0.01765 0.01795 0.01805
    ]

  Not so sure about this one so I've taken it away again.

17 July 2024.  Found a bug in KheDoSwap which could have had a
  material effect.  So here is the previous run again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 299.1 secs:
      0.01540 0.01610 0.01635 0.01645 0.01645 0.01685 0.01690 0.01715
      0.01725 0.01745 0.01845 0.01875
    ]

  Not a lot different from the second-last run yesterday.  Oh well.
  
  Now I've fixed things so that from_r's mtask set is only
  calculated once per interval and passed through to KheDoSwap.
  Here's the same run again after doing that:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 293.3 secs:
      0.01580 0.01605 0.01615 0.01615 0.01680 0.01680 0.01695 0.01715
      0.01755 0.01825 0.01880 0.01880
    ]

  Not a lot different, in fact somewhat worse - why?  Because the
  scratch mts is being clobbered as we go further down.  Fixed
  that problem by including a free list of mtask sets in ao,
  and then got this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 297.9 secs:
      0.01590 0.01610 0.01635 0.01635 0.01645 0.01670 0.01695 0.01705
      0.01715 0.01715 0.01780 0.01790
    ]

  Strange, the better my code gets, the worse the score gets.  But
  the worst result is far better than it was before.  Same again:
    
    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 297.1 secs:
      0.01620 0.01625 0.01635 0.01645 0.01690 0.01695 0.01695 0.01715
      0.01750 0.01755 0.01765 0.01790
    ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.4 secs:
      0.01605 0.01615 0.01615 0.01620 0.01665 0.01685 0.01685 0.01695
      0.01740 0.01745 0.01765 0.01790
    ]

  Why does it vary like this?  Could there be an uninitialized variable?

  Actually this optimization is not as effective as it promises to
  be.  So I'm withdrawing it now.  Then got this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 299.4 secs:
      0.01600 0.01645 0.01645 0.01665 0.01685 0.01690 0.01700 0.01715
      0.01725 0.01745 0.01760 0.01785
    ]

  Rel is 1.28.

  In current soln, why not 3Wed: NU_21 -> NU_24, 1Mon,1Tue: NU_24 ->NU_31.
  In fact why not just 1Mon,1Tue: NU_24 ->NU_31.  Must be history.  Yes:
  NU_24 has two free days off in history, so freeing 1Mon,1Tue would
  make 9 consecutive free days.

18 July 2024.  Generally spent time looking into the current defects,
  trying to find repairs for them.  No real progress but worth doing.

19 July 2024.  In KHE, there are 6 italics assigned to HN nurses.  In 
  LOR there are none.  That is a pretty stark difference, it implies 6
  overloads which cost 20 each.  And indeed there are 9 overloads among
  the HR nurses in LOR17, and 15 among them in KHE.

  Look at the italic Early HN_4 3Wed.  LOR17:

                  3Sat        3Sun
  ----------------------------------------
  Early             6 + 1U       5
  Day               6            7
  Late              5            4
  Night             4            5
  ----------------------------------------
  So (Late, Night) and (Early, Day) works and LOR does that.

                  3Sat        3Sun
  ----------------------------------------
  Early             7            5
  Day               6            7
  Late              5            4
  Night             4            5
  ----------------------------------------
  (Late, Night), (Early, Day), (Late, italic Late), (Early, Late)

20 July 2024.  Focussing on the 6 italics assigned to HN nurses
  in the KHE solution, since in the LOR solution there are none.

  What about fixing tasks that we've just moved, and unfixing
  them on unsuccessful return?  It might save a lot of time.
  But who is in charge of it?  Can we run along the path and
  fix the tasks assigned there?  Or even fix as we add them
  to the path?  Can we have a fix count?  Or do it ourselves
  as we assign?  The undo will unfix.  I tried this but got
  bogged down in a wilderness of cases where fixing one part
  of the repair meant that the rest could not be carried out.
  In any case it is not safe to fix a task when it is part
  of an mtask.

  Here is the current state:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01550 0.01625 0.01680 0.01680 0.01685 0.01690 0.01690 0.01700
      0.01760 0.01765 0.01790 0.01800
    ] 10 distinct costs, best soln (cost 0.01550) has diversifier 10

  Rel = 1.24.  I wish I knew why each run varies so much from the next.
  Here'a a run where every solve has the same diversifier (10):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 290.9 secs:
      0.01540 0.01540 0.01540 0.01540 0.01540 0.01540 0.01540 0.01565
      0.01665 0.01665 0.01665 0.01665
    ]

  There is a lot more similarity.  Now run it again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 290.6 secs:
      0.01555 0.01565 0.01565 0.01565 0.01565 0.01565 0.01565 0.01565
      0.01655 0.01655 0.01655 0.01655
    ]

  I did two single-threaded runs which both returned solutions with
  cost 1475 (pretty darn good, rel = 1.18).  This was with the
  diversifier set to 10.  Still have 4 italic shifts in the HNs.
  There were only slight differences in the calls made, according to diff:

    27c27
    < [queries   6305, cost_after    0.02450] rec
    ---
    > [queries   6301, cost_after    0.02450] rec
    45c45
    < [queries  83154, cost_after    0.01510] rdv
    ---
    > [queries  83149, cost_after    0.01510] rdv

  Presumably these are due to time limit issues - what else?
  Now taking the diversifier away we get a solution with cost 1715.
  What an incredibly large difference.

21 July 2024.  Wrote up an "assigning by history" section of the
  resource assignment chapter that is intended to replace the
  "grouping by history" section of the resource structural
  chapter.  Ready to implement; I can reuse much of grouping
  by history.

22 July 2024.  Implementing assigning by history.  I've audited
  the documentation, copied file khe_sr_group_by_history.c to new
  file khe_sr_assign_by_history.c, and done an initial boilerplate
  pass through khe_sr_assign_by_history.c.  Now for the real work.

23 July 2024.  Still implementing assigning by history.  Now handling
  multiple occurrences of a given resource in one class correctly.
  All done and ready to go, including adding "rah(<solver>)" to the
  do-it-yourself solver; this fixes the assignments of the tasks
  assigned by rah, and unfixes them after <solver> ends.

24 July 2024.  Testing assigning by history.  Found a few bugs but
  it seems to be working now; I need to evaluate the results it is
  getting.  I've now also killed off grouping by history.  Here
  are my first results:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 297.3 secs:
      0.01600 0.01605 0.01645 0.01650 0.01685 0.01755 0.01755 0.01760
      0.01760 0.01805 0.01830 0.01870
    ] 10 distinct costs, best soln (cost 0.01600) has diversifier 0

  The best solution here has 4 history defects, which seems good
  since LOR has 3.  But the overall result seems worse.  Here is
  a run without assign by history:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01540 0.01585 0.01600 0.01675 0.01685 0.01690 0.01715 0.01725
      0.01735 0.01745 0.01840 0.01875
    ] 12 distinct costs, best soln (cost 0.01540) has diversifier 10

  and the best solution here has 5 history defects.  Rel = 1.23.
  Here is a run with assign by history but without fixing:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01590 0.01615 0.01650 0.01655 0.01660 0.01660 0.01710 0.01730
      0.01740 0.01770 0.01780 0.01780
    ] 

  Just random differences?  It's interesting that the worst soln
  here has cost 100 less than the worst of the others.  I'll stick
  with fixing for now, it seems more in keeping with the idea.

  Off-site backup at this point.  Fresh start using rah:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.1 secs:
      0.01605 0.01645 0.01665 0.01685 0.01700 0.01700 0.01705 0.01745
      0.01755 0.01760 0.01770 0.01830
    ] 11 distinct costs, best soln (cost 0.01605) has diversifier 5

  The cost is not great, but this solution has the same 3 history
  defects that LOR17 has.  So it's got to be on the right track.
  There are 3 italic entries among the HN nurses, which is also
  an improvement, althouth LOR17 has none.

26 July 2024.  Yesterday I did some work on time limit consistency.
  Today I've got it all implemented, audited, and documented, and
  it's ready to test.  First test:

    [ "INRC2-4-100-0-1108", 1 solution, in 117.9 secs: cost 0.01705 ]
    [ "INRC2-4-100-0-1108", 1 solution, in 117.3 secs: cost 0.01705 ]
    [ "INRC2-4-100-0-1108", 1 solution, in 117.8 secs: cost 0.01705 ]

  It seems to be working.  Now for 12 solves in parallel:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.6 secs:
      0.01605 0.01645 0.01645 0.01685 0.01690 0.01700 0.01745 0.01760
      0.01760 0.01760 0.01805 0.01870
    ] 9 distinct costs, best soln (cost 0.01605) has diversifier 5

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 289.3 secs:
      0.01645 0.01645 0.01685 0.01690 0.01700 0.01745 0.01760 0.01760
      0.01805 0.01870 0.02690 0.02870
    ] 10 distinct costs, best soln (cost 0.01645) has diversifier 9

  Something is very wrong here.  Also, there are thousands of lines
  containing just 1.  What is causing this?

27 July 2024.  Looking into detailed behaviour of time limit
  consistency.  I worked out why there were so many "1" entries,
  it was just rdv and rec working themselves out when time ran
  out.  I've changed them both so that they don't do that now,
  and I've got it down to a .tlc file with just 9 lines.  So
  now for another real test:

    [ "INRC2-4-100-0-1108", 1 solution, in 281.8 secs: cost 0.01625 ]
    [ "INRC2-4-100-0-1108", 1 solution, in 280.7 secs: cost 0.01625 ]

  So far so good.

  I've sorted out what to do about TimeSetRemainingTime - just
  return KHE_NO_TIME when time limit consistency is active.  All
  done and documented and ready to test.

28 July 2024.  Made some adjustments to khe_sm_timer.c and started
  testing.  Here's the first test:

    [ "INRC2-4-100-0-1108", 1 solution, in 117.5 secs: cost 0.01710 ]
    [ "INRC2-4-100-0-1108", 1 solution, in 119.3 secs: cost 0.01710 ]
    [ "INRC2-4-100-0-1108", 1 solution, in 118.0 secs: cost 0.01710 ]

  And now a longer test:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.9 secs:
      0.01600 0.01610 0.01650 0.01655 0.01685 0.01735 0.01745 0.01755
      0.01760 0.01770 0.01830 0.01900
    ] 12 distinct costs, best soln (cost 0.01600) has diversifier 5

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.5 mins:
      0.01600 0.01610 0.01650 0.01655 0.01685 0.01735 0.01745 0.01755
      0.01760 0.01770 0.01830 0.01900
    ] 12 distinct costs, best soln (cost 0.01600) has diversifier 5

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.4 mins:
      0.01600 0.01610 0.01650 0.01655 0.01685 0.01735 0.01745 0.01755
      0.01760 0.01770 0.01830 0.01900
    ] 12 distinct costs, best soln (cost 0.01600) has diversifier 5

  Beautiful, it's working, and it proves that we don't have any
  uninitialized variables.  About three days to get here.

  Back to testing.  Using the cost 1600 (rel 1.28) solution here:

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       24
    O  Available times (negative)     43       56
    Y  Unnecessary assignments        19       27
    X  Unassigned tasks                5        0
    ---------------------------------------------
    U - O + Y - X                     -5       -5
  
  So the problem how seems to be Y (italic entries), where
  we are paying (27 - 19) * 20 = 160 unnecessarily.  If we
  could fix that we would be down to rel = 1.15.  Or if we
  could do as well on limit active intervals defects as LOR
  is doing, we would be down to 1.12.

  Of the 8 unnecessary italic entries, 3 are in the HN nurses.

  TR_89 --> TR_97 on 3Wed - why not?  Because of this:

    -Swap(3Wed, TR_89 {3Wed:Late.35 [3Wed]}, TR_97 {3Wed:Late.35 [3Wed]},
      3Wed3): (b blocking 3Wed:Late.35)

  The trouble is that no-one has noticed that the TR_97 task is
  optional and does not need to be swapped back again.  It should be
  unassigned.  I tried to fix this by adding unassign_extreme_unneeded
  to KheFindReassignableMTasksInInterval, but look what I got:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.9 secs:
      0.01665 0.01685 0.01710 0.01720 0.01725 0.01730 0.01745 0.01800
      0.01800 0.01810 0.01840 0.01855
    ] 11 distinct costs, best soln (cost 0.01665) has diversifier 5

  Here is the same run (without tlc) again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.7 secs:
      0.01665 0.01690 0.01695 0.01705 0.01720 0.01725 0.01770 0.01775
      0.01775 0.01800 0.01815 0.01880
    ] 11 distinct costs, best soln (cost 0.01665) has diversifier 5

  So I've taken away this "improvement".  I need to work out more
  carefully what to do here.  So back to previous state:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.9 secs:
      0.01575 0.01605 0.01665 0.01700 0.01705 0.01735 0.01755 0.01770
      0.01770 0.01795 0.01805 0.01890
    ] 11 distinct costs, best soln (cost 0.01575) has diversifier 0

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.5 mins:
      0.01575 0.01605 0.01665 0.01700 0.01705 0.01735 0.01755 0.01770
      0.01770 0.01795 0.01805 0.01890
    ] 11 distinct costs, best soln (cost 0.01575) has diversifier 0

  Another successful test of tlc.  But I've lost the solution in
  which whynot TR_89 --> TR_97 on 3Wed occurred, so I can't work on
  that.  Still have 3 HN italics, but they are all on Sundays.

29 July 2024.  Looking at total workload:

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       57
    Y  Unnecessary assignments        19       26
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  So the excess cost caused by this is (57 - 43 + 1 - 5) = 10,
  times cost 20 is 200.  The soln difference is more like 300
  because it includes another 60 for weekend overloads.

30 July 2024.  Changed monitor adjustment into solution adjustment,
  and added fixing and unfixing tasks as adjustments.  All implemented
  and documented.  I'm ready to return to unbalanced complete weekends
  now, and I'll be able to decide what to do without being biased by
  what solution adjustment can do.

  I've reorganized the code and documentation to move as much as
  possible out of "solution adjustment" in general solvers and into
  "solution adjustments" in resource-structural solvers.

  Started work on khe_sm_balance_weekends.c, only boilerplate so far.

31 July 2024.  Working on khe_sm_balance_weekends.c  Just finished
  building equivalence classes of complete weekends constraints.
  Need some debug code and then test what I've done so far.

  Reorganized resource yourself so that the items appear in the
  same order that they appear in the Guide.  And added balance
  weekends.

1 August 2024.  Working on khe_sm_balance_weekends.c.  The last bit,
  working out costs and comparing them, needs an audit.  Otherwise
  all done and ready to test.

2 August 2024.  Working on khe_sm_balance_weekends.c.  Audited the
  documentation and the implementation and started testing.  The first
  test fixed 1Sun, 2Sun, 3Sun, and 4Sun, which is correct I believe.
  First result is

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.0 secs:
      0.01605 0.01665 0.01675 0.01700 0.01715 0.01725 0.01735 0.01735
      0.01770 0.01775 0.01780 0.01825
    ]

  And another go, without consistency:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.0 secs:
      0.01600 0.01605 0.01615 0.01655 0.01675 0.01700 0.01720 0.01740
      0.01740 0.01775 0.01780 0.01850
    ]

  These results are not great, but we need to look into the details.

  Tried running with fixed tasks for the whole run.  This broke
  dynamic, so I added KheMTaskFixedAndUnassigned and skipped
  mtasks that satisfy this condition.

3 August 2024.  Fixed dynamic by changing unassigned_tasks into
  available_tasks and not adding fixed unassigned tasks to it.
  But the result of a full run with fixed tasks was quite poor:

    [ "INRC2-4-100-0-1108", 1 solution, in 279.4 secs: cost 0.01935 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 294.0 secs:
      0.01920 0.01955 0.01965 0.01985 0.01995 0.02005 0.02025 0.02035
      0.02045 0.02055 0.02070 0.02095
    ]

  So let's go back to the original intention and fix for just
  the first phase:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 295.9 secs:
      0.01605 0.01675 0.01675 0.01675 0.01695 0.01700 0.01725 0.01740
      0.01745 0.01755 0.01775 0.01845
    ]

  It's not a bad result, but there is no real improvement over
  not fixing.  Italic entries are 28, which is two more than
  the 26 I was getting before.

  Here's a run where we balance history on the first two
  stages, unfixing only for the last stage:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.3 secs:
      0.01655 0.01670 0.01675 0.01680 0.01730 0.01745 0.01745 0.01755
      0.01755 0.01770 0.01785 0.01810
    ]

  I've decided to not use balancing weekends, at least for now,
  but to think about it off-line and maybe return to it later.
  It hasn't produced the good results I was hoping for, and as
  currently implemented it does not allow both times on the
  weekend to be italic.

  Here we are without balancing weekends:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.8 secs:
      0.01585 0.01645 0.01685 0.01685 0.01700 0.01720 0.01730 0.01755
      0.01760 0.01770 0.01795 0.01825
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       24
    O  Available times (negative)     43       55
    Y  Unnecessary assignments        19       26
    X  Unassigned tasks                5        0
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  TR_84 1Wed Late --> {} whynot?  Because TR_84 is half-time, and
  its busy days have to come in sequences of 3 to 5.

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 296.0 secs:
      0.01600 0.01605 0.01645 0.01685 0.01690 0.01720 0.01755 0.01760
      0.01760 0.01795 0.01825 0.01870
    ]

  Things seem to have got a bit worse, not sure why.  Fiddled
  with time limit apportionment and go again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01590 0.01600 0.01605 0.01620 0.01630 0.01670 0.01700 0.01720
      0.01735 0.01735 0.01755 0.01830
    ]

  1590 is rel = 1.27.

4 August 2024.  Back at work on KheBalanceWeekends, adding two options,
  rs_balance_weekends_method and rs_balance_weekends_no_undo.  I've
  documented and implemented both, ready for testing, although the code
  for rs_balance_weekends_method="khe_balance_weekends_fix_required"
  needs an audit.

5 August 2024.  Audited khe_balance_weekends_fix_required and started
  testing.  First results:

    [ "INRC2-4-100-0-1108", 1 solution, in 278.9 secs: cost 0.01645 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01570 0.01625 0.01635 0.01650 0.01655 0.01675 0.01695 0.01695
      0.01715 0.01760 0.01810 0.01855
    ] 11 distinct costs, best soln (cost 0.01570) has diversifier 6

  This is slightly better than I have been getting without it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 299.9 secs:
      0.01590 0.01605 0.01625 0.01685 0.01685 0.01695 0.01700 0.01740
      0.01755 0.01775 0.01785 0.01810
    ]

  although the differences may just be random.

  Found out that I was including all required tasks, not just tasks
  of the current resource type.  Corrected that and I'm now getting
  the right number of tasks being fixed.  So that's good.  Results
  are below.  They are They is reasonable, not brilliant.  With no_undo:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 298.8 secs:
      0.01615 0.01650 0.01650 0.01665 0.01690 0.01700 0.01715 0.01720
      0.01775 0.01805 0.01820 0.01825
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       25
    O  Available times (negative)     43       47
    Y  Unnecessary assignments        19       23
    X  Unassigned tasks                5        6
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  Not a great result, although it is somewhat closer to LOR.  So
  I will abandon no_undo.  Here is a 15-minute run without it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 14.0 mins:
      0.01570 0.01575 0.01600 0.01620 0.01625 0.01635 0.01660 0.01665
      0.01670 0.01685 0.01690 0.01690
    ]

  The best result is no better than before, although the worst is a
  lot better.  Here is the 5-minute best of 12 that I'm working from:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01570 0.01600 0.01645 0.01650 0.01675 0.01695 0.01725 0.01725
      0.01795 0.01795 0.01800 0.01870
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       55
    Y  Unnecessary assignments        19       25
    X  Unassigned tasks                5        3
    ---------------------------------------------
    U - O + Y - X                     -5       -5


  I found a defect which suggested to me that I should still be
  doing *ejecting* move repairs - they had been turned off.  After
  turning them back on I got this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 298.4 secs:
      0.01585 0.01600 0.01620 0.01660 0.01675 0.01690 0.01720 0.01725
      0.01775 0.01780 0.01785 0.01830
    ]

  It's turned out slightly worse, presumably because time was wasted
  on repairs that were not useful.  Taken it away again.

6 August 2024.  Making a fresh start based on this run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01570 0.01600 0.01645 0.01670 0.01685 0.01685 0.01690 0.01700
      0.01725 0.01790 0.01795 0.01820
    ] 11 distinct costs, best soln (cost 0.01570) has diversifier 11

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       55
    Y  Unnecessary assignments        19       25
    X  Unassigned tasks                5        3
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  which has rel = 1.26.

  Assign by history is working well but its results don't seem
  to be sticking.  So I've tried fixing them permanently.  Result is

  [ "INRC2-4-100-0-1108", 1 solution, in 280.4 secs: cost 0.01570 ]

  Sorted out fixing in assign by history, now I can vary it with
  option rs_assign_by_history_fix.  Here is the result for each
  of the three values:

  "none" (result has rel = 1.23):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01600 0.01655 0.01675 0.01695 0.01715 0.01735 0.01755
      0.01770 0.01800 0.01800 0.01800
    ]

  "fix"

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01600 0.01605 0.01650 0.01650 0.01675 0.01695 0.01700 0.01710
      0.01735 0.01735 0.01810 0.01885
    ]

  "fix_no_undo" (actually found on 7 August after fixing the abort):

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01775 0.01780 0.01795 0.01795 0.01815 0.01820 0.01820 0.01825
      0.01825 0.01835 0.01845 0.01875
    ]

7 August 2024.  Sorted out the KheTaskerGroupingBuild abort problem
  by not including fixed tasks.  All done and documented.  Looking
  at the results (from yesterday), it seems that not fixing is best,
  and better than doing nothing about history.  Here is running
  with assign by history off:

  [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
    0.01595 0.01630 0.01630 0.01635 0.01645 0.01650 0.01660 0.01665
    0.01695 0.01770 0.01780 0.01825
  ]

  and here with assign by history on and fix set to none:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01565 0.01585 0.01600 0.01655 0.01660 0.01685 0.01705
      0.01730 0.01730 0.01755 0.01800
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       56
    Y  Unnecessary assignments        19       26
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  There is definitely an improvement here.  As stated, rel = 1.23.
  But we are still 10 behind on U + Y, which equates to 200 points.
  If we could get rid of them we would be laughing (rel = 1.08).
  But what we have achieved here has more to do with limit active
  intervals constraints:  their cost is down to 135, only 60 more
  than LOR's 75.  And the history cost is 90 compared with LOR's 75.

  The KHE solution has 4 italic tasks in the HN part, LOR has none,
  although it has 3 more unassigned tasks than the KHE solution.

8 August 2024.  Whynot full cycle CT_45 <--> CT_67?  It should work,
  but there is the whynot run, it does not seem to have noticed that
  the overall cost has improved.  There was a history problem which
  meant that the full timetable swap did not improve things.

  Big blowout in max working weekends.  KHE has cpst 210 for them,
  LOR has cost 90.

  Thinking about a rematch of half the cycle at the end.  Meanwhile,
  here is what happens if you add rrm at the end:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 277.1 secs:
      0.01535 0.01570 0.01580 0.01585 0.01665 0.01675 0.01690 0.01710
      0.01720 0.01725 0.01785 0.01795
    ]

  Amazing, it actually helped (rel = 1.23), although it blew out
  history limit active intervals defects again - no, not true,
  they blew out in the 1545 solution as well.  So let's go on
  and try rematching half the cycle.  Here is what happens if you
  add rrm at the end of the first repair phase:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01600 0.01645 0.01665 0.01680 0.01715 0.01750 0.01755 0.01765
      0.01770 0.01785 0.01825 0.01825
    ]

  Not nearly so good, that's interesting.  Removed rrm but gave
  more time to the first repair phase:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01545 0.01580 0.01580 0.01660 0.01680 0.01685 0.01685 0.01720
      0.01725 0.01735 0.01755 0.01790
    ]

  So back to rrm at the end and weight 2 to the first repair phase:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 277.2 secs:
      0.01535 0.01580 0.01645 0.01650 0.01655 0.01660 0.01665 0.01675
      0.01720 0.01755 0.01770 0.01770
    ]

9 August 2024.  On trainees, KHE compares pretty well:

    TRAINEES ONLY                    LOR      KHE
    ---------------------------------------------
    U  Available times (positive)      4        5
    O  Available times (negative)      7       10
    Y  Unnecessary assignments        18       19
    X  Unassigned tasks                1 (?)    0 (?)
    ---------------------------------------------
    U - O + Y - X                     14       14

  Thinking about grouping.  Here is a run with grouping off:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 277.7 secs:
      0.01605 0.01620 0.01665 0.01670 0.01675 0.01680 0.01695 0.01710
      0.01730 0.01735 0.01765 0.01765
    ]

  Somewhat worse.  Anyway we need grouping for COI instances.
  Tried with balancing weekends off, got 1565, slightly worse.
  Back to our starting point:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 278.4 secs:
      0.01535 0.01570 0.01640 0.01645 0.01650 0.01660 0.01665 0.01675
      0.01685 0.01745 0.01795 0.01840
    ] 12 distinct costs, best soln (cost 0.01535) has diversifier 11

  So let's see what grouping we get with diversifier 11.  There is a
  lot of grouping, including nights - about 600 groups.   Non-night
  shifts are main two's with some three's; night shifts are fours
  and fives.  These are the groupings made for head nurses on night
  shifts in the first week:

    1Fri:Night.0{1Sat:Night.0{}, 1Sun:Night.0{}, 2Mon:Night.0{}} (HeadNurse)
    1Tue:Night.0{1Wed:Night.0{}, 1Thu:Night.3{}, 1Fri:Night.5{}} (HeadNurse)
    1Mon:Night.0{1Tue:Night.5{}, 1Wed:Night.5{}, 1Thu:Night.4{}} (HeadNurse)

  These three are repeated once each in the listing, not sure why.
  Briefly they are 1Fri-2Mon, 1Tue-1Fri, 1Mon-1Thu.  All have length 4.
  They all appear in the KHE solution, but none of them appear in the
  LOR solution (except there is a 1Mon-1Fri).  So does task grouping
  need to be rethought to include history?  Or should task grouping
  hold off during the first week?  Here are the history assignments:

    assigning HN_4 to task 1Mon:Day.13{} (no fix)
    assigning HN_1 to task 1Mon:Night.11{} (no fix)
    assigning HN_8 to task 1Mon:Night.0{} (no fix)
    assigning HN_1 to task 1Tue:Night.11{} (no fix)
    assigning HN_8 to task 1Tue:Night.0{} (no fix)
    assigning HN_8 to task 1Wed:Night.0{} (no fix)

  They precede the grouping assignments in the run, which is right,
  but they seem to have been ignored by grouping, which is wrong.

  I tried a couple of stupid things but really the only option
  is to fix these assignments.  Then grouping will ignore them.
  Here's a run with rs_assign_by_history_fix=fix:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 274.7 secs:
      0.01655 0.01665 0.01700 0.01705 0.01705 0.01710 0.01715 0.01735
      0.01750 0.01770 0.01780 0.01785
    ] 11 distinct costs, best soln (cost 0.01655) has diversifier 4

  This has 6 history defects with total cost 15+15+45+30+30+30 = 165.
  Here's a run with rs_assign_by_history_fix=none:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 277.7 secs:
      0.01535 0.01565 0.01580 0.01675 0.01680 0.01680 0.01705 0.01705
      0.01720 0.01755 0.01770 0.01795
    ]

  This has 5 history defects with total cost 15+15+45+15+30 = 120.
  Now rs_assign_by_history_fix=none without rdv:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 236.6 secs:
      0.01580 0.01615 0.01650 0.01670 0.01685 0.01705 0.01705 0.01725
      0.01785 0.01790 0.01800 0.01800
    ]

   Not as good as with rdv, but it's hard to be sure.  It's faster.

10 August 2024.  Start again with rs_assign_by_history_fix=fix:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 275.1 secs:
      0.01580 0.01600 0.01625 0.01655 0.01705 0.01725 0.01735 0.01755
      0.01760 0.01780 0.01800 0.01800
    ] 11 distinct costs, best soln (cost 0.01580) has diversifier 8

  Here are the assign by history results (HN night shifts only):

    assigning HN_1 to task 1Mon:Night.11{} (fix)
    assigning HN_8 to task 1Mon:Night.0{} (fix)
    assigning HN_1 to task 1Tue:Night.11{} (fix)
    assigning HN_8 to task 1Tue:Night.0{} (fix)
    assigning HN_8 to task 1Wed:Night.0{} (fix)

  Not sure why 1Mon:Night.11 was chosen, it is

    <R>A=s0+NWNurse=h1:5</R>

  when a much better choice would have been 1Mon:Night.5:

    <R>NA=h1|NWNurse=h1:1</R>

  This is borne out by the debug output too:

    edge from HN_1 to 1Mon:Night.5: true (rn asst_cost 0.00000,
      rn non_asst_cost 0.00030, tn asst_cost 0.00000, tn non_asst_cost 1.00000)
    edge from HN_1 to 1Mon:Night.11: true (rn asst_cost 0.00000,
      rn non_asst_cost 0.00030, tn asst_cost 0.00000, tn non_asst_cost 0.00000)

  The match should have gone for 1Mon:Night.5 over 1Mon:Night.11, but

    KheAssignByHistory assigning HN_1 to 1Mon:Night.11{} (fix)
    KheAssignByHistory assigning HN_8 to 1Mon:Night.0{} (fix)

  is what we actually got.

  OK, I've fixed it.  I was miscalculating edge costs.  I'm now
  getting the history assignments I wanted all along.  So here
  is a full solve with rs_assign_by_history_fix=fix:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 279.5 secs:
      0.01640 0.01655 0.01670 0.01695 0.01695 0.01705 0.01705 0.01705
      0.01715 0.01730 0.01765 0.01785
    ] 9 distinct costs, best soln (cost 0.01640) has diversifier 8

  Not a great result.  I should look over this carefully to work
  out why.  Meanwhile, with rs_assign_by_history_fix=none:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 270.2 secs:
      0.01470 0.01650 0.01670 0.01680 0.01685 0.01690 0.01695 0.01730
      0.01780 0.01790 0.01800 0.01835
    ] 12 distinct costs, best soln (cost 0.01470) has diversifier 10

  It's an outlier but for a 5-minute run it is a new best, rel=1.18.
  With rs_assign_by_history_fix=fix_no_undo:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 277.0 secs:
      0.01700 0.01705 0.01710 0.01720 0.01730 0.01790 0.01790 0.01795
      0.01810 0.01830 0.01845 0.01865
    ] 11 distinct costs, best soln (cost 0.01700) has diversifier 4

  Let's go back now to rs_assign_by_history_fix=fix and investigate
  its performance in detail.  Here is is again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 278.8 secs:
      0.01575 0.01635 0.01645 0.01655 0.01670 0.01685 0.01690 0.01705
      0.01755 0.01770 0.01785 0.01805
    ] 12 distinct costs, best soln (cost 0.01575) has diversifier 6

  The history result here is pretty good, total 90 compared with 75
  for LOR.  The gap was in HN_8 where LOR falls one short but KHE
  falls two short.  Here are the assignments made by history in the
  1575 solution (HN night only):

    KheAssignByHistory assigning HN_1 to 1Mon:Night.5{} (fix)
    KheAssignByHistory assigning HN_8 to 1Mon:Night.0{} (fix)
    KheAssignByHistory assigning HN_1 to 1Tue:Night.6{} (fix)
    KheAssignByHistory assigning HN_8 to 1Tue:Night.0{} (fix)
    KheAssignByHistory assigning HN_8 to 1Wed:Night.0{} (fix)

  These are also the assignments that LOR ends up with (although
  the Wed shift goes to HN_1), but they get moved around subsequently
  in KHE.  Let's see what profile grouping is doing here:

    backward: 1Fri:Night.0{1Sat:Night.0, 1Sun:Night.0, 2Mon:Night.0} (HeadNurse)
    backward: 1Tue:Night.5{1Wed:Night.5, 1Thu:Night.3, 1Fri:Night.5} (Nurse)

  Only these two groups for head nurses and nurses in the night shifts
  in the first week.  It's not much.  Neither of these groups appears
  in the LOR solution, except that LOR does have 1Mon-1Fri, given to a
  nurse, so that presumably covers it.  I don't see any reason why
  a HN run should end on 2Mon.  I need to look into it in detail.

  I think it's because the grouping treats each different domain
  in isolation, but all the head nurses are taking many nurse
  jobs as well as head nurse jobs.  So arguably we should treat
  head nurse and nurse as largely interchangeable, we should see
  how much overlap there is between these two resource groups.
  Yes, every HeadNurse is also a Nurse, so grouping specifically
  on HeadNurse does not make sense.

  So here's a run with assign by history in good shape but with
  grouping turned off:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 285.7 secs:
      0.01585 0.01625 0.01665 0.01710 0.01710 0.01720 0.01725 0.01750
      0.01750 0.01775 0.01805 0.01950
    ] 10 distinct costs, best soln (cost 0.01585) has diversifier 1

  There is not a lot of difference from grouping on, but we
  can't turn grouping off because of the COI instances.

  Started looking through the doc in preparation for revising
  grouping by resource constraints.  It probably does not
  matter that tasks with different domains go into different
  classes.  Profile grouping works off the number of tasks
  in all classes.  But then why does it find the two cases
  above?  We need to print the profile and see what's in it.

11 August 2024.  Looking into what profile grouping is doing.

  * Profile grouping passes KHE_COMB_SOLVER_COST_ZERO to the
    comb solver.  Why isn't this KHE_COMB_SOLVER_COST_SOLE_ZERO?

  * It passes time groups <no, yes, yes, yes, -, -, no> where
    - means nothing at all.  This seems wrong, surely it should
    pass <no, yes, yes, yes, free, free, no>?

  Worked on a "resource flow" section of the doc.  The idea
  is to use it to find sets of resources more suited to use as the
  domains in profile grouping than the resource groups I'm using now.

12 August 2024.  Working on khe_sr_resource_flow.c.  I'm now building
  the resource nodes and task nodes and adding edges between them and
  merging equivalent resource nodes.  It needs some debug printing,
  then I will be ready to add a flow graph and start solving.

13 August 2024.  Working on khe_sr_resource_flow.c.  I've now got
  the graph I want.

14 August 2024.  Working on khe_sr_resource_flow.c.  Got this flow:

    [ Flow
      FlowResourceNode(16 HeadNurse, 196 capacity)
      --75-> FlowTaskNode 0(91 tasks, 91 capacity)
      --121-> FlowTaskNode 1(239 tasks, 239 capacity)
      FlowResourceNode(19 resources, 250 capacity)
      --118-> FlowTaskNode 1(239 tasks, 239 capacity)
      --132-> FlowTaskNode 2(669 tasks, 669 capacity)
      FlowResourceNode(40 resources, 537 capacity)
      --537-> FlowTaskNode 2(669 tasks, 669 capacity)
    ]

  I've checked it, it's correct.  I've also audited the module and
  got it into very clean order.  But what now?

15 August 2024.  Thinking about the effect of the flow graph on
  profile grouping.

  Investigated possible confusion between KheResourceMaxBusyTimes
  and KheResourceAvailableBusyTimes, in documentation and code.
  And I did indeed find some confusion in both, including making
  one change to the code.  All fixed now, must take care in future.

  Prepared my PATAT slides and did a run through - it took 40
  minutes, so I need to go faster, but there is plenty there.

16 August 2024.  Did another run through, got it down to 30 mins.

  Each time we group some tasks, remove them from the flow graph.
  Make a guess as to which kind of resource will be assigned to
  them and reduce its capacity as well.  Then if the maximum flow
  reduces, that grouping was not a good idea.  Or we could use
  the global tixel matching and reduce the domains of the grouped
  tasks to the intersection of their domains.  Indeed that might
  happen automatically right now.

  If the total number of nurses changes, that proves that a
  sequence begins or ends there.  But it says nothing about
  which kind of nurse.  Surely if we are going to use a
  HeadNurse task we would prefer the other tasks to also
  be HeadNurse tasks - but thay may not be viable, in which
  case we can go to some other kind of task in order to get
  the grouping.

17 August 2024.  Finished off the overheads for my PATAT talk,
  posted them on my web site and put them on a USB stick.

  Pondered the "varying task domains" section on page 46.  It
  sounds plausible but when you compare it with the reality of
  head nurses and ordinary nurses it starts to wobble.  The
  profile proves that some nurse must end a sequence there,
  but it could be a head nurse or an ordinary nurse.

  Looking at the flow graph on p19, caretakers can take only
  caretaker shifts.  Is that more amenable?

  Is profile grouping really a flow model?  Is each group really
  one flow?  How do we force min 3, no HN+Caretaker, etc? 

  What about this?  We find all tasks that HN_* can be assigned
  to, and we find profiles among them.  No, that isn't right.

18 August 2024.  Done an experiment to see whether profile grouping
  correlates with what LOR is coming up with.  Here are all of the
  night shift groupings for trainees:

    combinatorial:
    to a] 1Sat:Night.19{1Sun:Night.19}
    to b] 2Sat:Night.19{2Sun:Night.19}
    to c] 3Sat:Night.17{3Sun:Night.19}
    to d] 3Sat:Night.18{3Sun:Night.20}

    forward:
    a] 1Wed:Night.32{1Thu:Night.31, 1Fri:Night.29, 1Sat:Night.19{1Sun:Night.19}}
    2Mon:Night.31{2Tue:Night.29, 2Wed:Night.31, 2Thu:Night.31}
    2Mon:Night.32{2Tue:Night.30, 2Wed:Night.32, 2Thu:Night.32}
    b] 2Wed:Night.33{2Thu:Night.33, 2Fri:Night.29, 2Sat:Night.19{2Sun:Night.19}}
    to g] 3Mon:Night.27{3Tue:Night.25, 3Wed:Night.29, 3Thu:Night.33}
    3Mon:Night.28{3Tue:Night.26, 3Wed:Night.30, 3Thu:Night.34}
    to e] 4Mon:Night.35{4Tue:Night.27, 4Wed:Night.31, 4Thu:Night.29}
    to f] 4Mon:Night.36{4Tue:Night.28, 4Wed:Night.32, 4Thu:Night.30}

    backward:
    e] 4Mon:Night.35{4Tue:Night.27, 4Wed:Night.31, 4Thu:Night.29, 4Fri:Night.29}
    f] 4Mon:Night.36{4Tue:Night.28, 4Wed:Night.32, 4Thu:Night.30, 4Fri:Night.30}
    c] 3Sat:Night.17{3Sun:Night.19, 4Mon:Night.37, 4Tue:Night.29, 4Wed:Night.33}
    d] 3Fri:Night.29{3Sat:Night.18{3Sun:Night.20}, 4Mon:Night.38}
    g] 3Mon:Night.27{3Tue:Night.25, 3Wed:Night.29, 3Thu:Night.33, 3Fri:Night.30}

    So the final groups are:
    a] 1Wed:Night.32{1Thu:Night.31, 1Fri:Night.29, 1Sat:Night.19{1Sun:Night.19}}
    2Mon:Night.31{2Tue:Night.29, 2Wed:Night.31, 2Thu:Night.31}
    2Mon:Night.32{2Tue:Night.30, 2Wed:Night.32, 2Thu:Night.32}
    b] 2Wed:Night.33{2Thu:Night.33, 2Fri:Night.29, 2Sat:Night.19{2Sun:Night.19}}
    3Mon:Night.28{3Tue:Night.26, 3Wed:Night.30, 3Thu:Night.34}
    e] 4Mon:Night.35{4Tue:Night.27, 4Wed:Night.31, 4Thu:Night.29, 4Fri:Night.29}
    f] 4Mon:Night.36{4Tue:Night.28, 4Wed:Night.32, 4Thu:Night.30, 4Fri:Night.30}
    c] 3Sat:Night.17{3Sun:Night.19, 4Mon:Night.37, 4Tue:Night.29, 4Wed:Night.33}
    d] 3Fri:Night.29{3Sat:Night.18{3Sun:Night.20}, 4Mon:Night.38}
    g] 3Mon:Night.27{3Tue:Night.25, 3Wed:Night.29, 3Thu:Night.33, 3Fri:Night.30}

  Or more briefly, the groups are these, with LOR assignments alongside:

    KHE Group   LOR asst
    ----------------------------------------------
    1Wed-1Sun   TR_91
    2Mon-2Thu   TR_76
    2Mon-2Thu   TR_93
    2Wed-2Sun   TR_82
    3Mon-3Thu   TR_92
    4Mon-4Fri   TR_96
    4Mon-4Fri   TR_93 + one unassigned night shift
    3Sat-4Wed   TR_83 3Sat-4Tue
    3Fri-4Mon   TR_84
    3Mon-3Fri   TR_88
    ----------------------------------------------

  That's amazing, there is a virtually perfect correlation between
  the groups that KHE is making and the assignments in the LOR solution.
  There are four or five sequences in the LOR solution that were not
  made into groups by KHE, but still the result is really wonderful.
  The problem is that we don't do nearly this well when nurse skills
  muddy the picture.

19 August 2024.  Another experiment looking at what grouping by
  resource constraints, together with assign by history, produce
  for HeadNurse night shifts:

  KheAssignByHistory assigning HN_1 to 1Mon:Night.5{} (fix)
  KheAssignByHistory assigning HN_8 to 1Mon:Night.0{} (fix)
  KheAssignByHistory assigning HN_1 to 1Tue:Night.6{} (fix)
  KheAssignByHistory assigning HN_8 to 1Tue:Night.0{} (fix)
  KheAssignByHistory assigning HN_8 to 1Wed:Night.0{} (fix)

  forward: 2Tue:Night.0{2Wed:Night.0, 2Mon:Night.5, 2Thu:Night.3}
  forward: 3Wed:Night.0{3Thu:Night.0, 3Mon:Night.3, 3Tue:Night.3}
  forward: 3Sun:Night.0{4Mon:Night.0, 4Tue:Night.0, 4Wed:Night.0}
  forward: 4Mon:Night.1{4Tue:Night.1, 4Wed:Night.5, 4Thu:Night.3}
  backward: 4Mon:Night.1{4Tue:Night.1, 4Wed:Night.5, 4Thu:Night.3, 4Fri:Night.5}
  backward: 3Wed:Night.0{3Thu:Night.0, 3Mon:Night.3, 3Tue:Night.3, 3Fri:Night.0}
  backward: 2Fri:Night.0{2Sat:Night.0, 2Sun:Night.0, 3Mon:Night.5}
  backward: 2Tue:Night.0{2Wed:Night.0, 2Mon:Night.5, 2Thu:Night.3, 2Fri:Night.5}
  backward: 1Fri:Night.0{1Sat:Night.0{}, 1Sun:Night.0{}, 2Mon:Night.0{}}

  In other words, for night shifts:

    Type       Interval     KHE    LOR                Hist requires
    ---------------------------------------------------------------
    History    1Mon-1Tue    HN_1   1Mon-1Wed HN_1     2 or 3
    History    1Mon-1Wed    HN_8   1Mon-1Tue HN_8     3 or 4
    Grouping   2Mon-2Fri           2Mon-2Fri HN_15
    Grouping   3Mon-3Fri           3Mon-3Fri HN_15
    Grouping   3Sun-4Wed           -- no similar --
    Grouping   4Mon-4Thu           4Mon-4Fri HN_15
    Grouping   2Fri-3Mon           2Fri-3Mon HN_4
    Grouping   1Fri-2Mon           -- no similar --
    ---------------------------------------------------------------

  It's not bad but it's not as compelling as the Trainee result, and
  that is because HN resources are free to take on (and indeed must
  take on) Nurse roles as well as HeadNurse roles.  We want to be able
  to justify the groups we make, and we can't really do that here.

  This is with rs_assign_by_history_fix=fix:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 279.0 secs:
      0.01575 0.01600 0.01635 0.01655 0.01670 0.01695 0.01705 0.01705
      0.01765 0.01785 0.01840 0.01895
    ]

  Here 1575 is rel = 1.26.  And with rs_assign_by_history_fix=none:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 271.2 secs:
      0.01490 0.01650 0.01655 0.01670 0.01680 0.01680 0.01685 0.01690
      0.01780 0.01795 0.01800 0.01855
    ]

  For 1490, rel = 1.19.
  But if you don't fix, what do the two assignment algorithms do?
  Do they just unassign and forget about you?  The dynamic solver
  certainly does.  Time Sweep has a rs_time_sweep_preserve_existing_off
  option which unassigns, but by default its value is false so
  by default it does preserve existing assignments.

  I need the equivalent of preserve_existing for sequential.
  And does time sweep really preserve existing, I mean what
  about the ejection chain calls?

20 August 2024.  Looking into preserve_existing parameter of
  KheResourceMatchingDemandSetMake.  Here are the spots where
  this function is called, and the values of the parameter:

    khe_sr_rematch.c:637         false
    khe_sr_time_sweep.c:1055	 !rs_time_sweep_preserve_existing_off
    khe_sr_time_sweep.c:1074     !rs_time_sweep_preserve_existing_off
    khe_sr_time_sweep.c:1105     false
    khe_sr_time_sweep.c:1167     !rs_time_sweep_preserve_existing_off

  So by default it is used by most of time sweep.  The exception
  there is rematching.

  Decided to remove preserve_existing, because (a) it does not
  work when ejection chains are invoked, and (b) it can be done
  properly by fixing tasks.  All done and documented.  Now for some
  tests.

  With rs_assign_by_history_fix=fix:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 282.1 secs:
      0.01560 0.01590 0.01650 0.01655 0.01665 0.01680 0.01705 0.01710
      0.01715 0.01725 0.01755 0.01765
    ] 12 distinct costs, best soln (cost 0.01560) has diversifier 5

  This is better; the worst cost is a lot better.
  With rs_assign_by_history_fix=none:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 273.8 secs:
      0.01610 0.01640 0.01645 0.01660 0.01695 0.01720 0.01720 0.01760
      0.01760 0.01785 0.01820 0.01825
    ]

  This is worse, but the 1490 solution I was getting before was always
  an outlier.  So I will go on with rs_assign_by_history_fix=fix.
  So let's resume the grind with that 1560 solution.

  I've decided to continue with rs_group_by_rc_off=false and not
  to change the grouping algorithm.  It's true that it is not well
  justified when there are skills, e.g. in INRC2-4-100-0-1108, but
  it works well sometimes and I can't see a good way to adjust it:

    "Working on khe_sr_resource_flow.c.  It's done and finding flows.
    But what now?  We did it to help decide what profile grouping
    we should undertake.  This example does suggest that grouping
    tasks with domain HeadNurse alone is not a reasonable thing to
    do, because in practice over half of the workload of HN_*
    resources (196 - 91 = 101 shifts) is given to Nurse tasks.
    But does that mean we should do nothing at all?  We don't want
    to turn off grouping altogether - see the wonderful results
    we are getting for Trainee night shifts (18 August 2024)."

  Done another run with rs_assign_by_history_fix=fix, this time
  turning on time limit consistency:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 281.7 secs:
      0.01590 0.01605 0.01610 0.01680 0.01685 0.01690 0.01695 0.01715
      0.01725 0.01730 0.01770 0.01955
    ] 12 distinct costs, best soln (cost 0.01590) has diversifier 3

  Not a great starting point (rel = 1.27), but anyway.  Let's have a look at
  history.  Here's LOR's history costs:

    Constraint Id / Point 	Calculation 			Inf.   Obj.
    -----------------------------------------------------------------------
    Constraint:17/HN_8 	15 * Linear(4 - History 1 - 1Mon4 - 1Tue4)    	15
    Constraint:17/TR_88 	15 * Linear(4 - History 1 - 1Mon4)    	30
    Constraint:17/TR_98 	15 * Linear(4 - History 2) 	   	30
    -----------------------------------------------------------------------
	  Total (3 points)						75 

  And here's KHE's history costs (non-history omitted):

    Constraint Id / Point 	Calculation 			Inf. 	Obj.
    -----------------------------------------------------------------------
    Constraint:17/HN_8 	15 * Linear(4 - History 1 - 1Mon4) 	   	30
    Constraint:17/TR_88 	15 * Linear(4 - History 1 - 1Mon4)    	30
    Constraint:17/TR_92 	15 * Linear(4 - History 2 - 1Mon4)    	15
    Constraint:17/TR_98 	15 * Linear(4 - History 2) 	   	30
    -----------------------------------------------------------------------
          Total (4 points) 			   	   	       105

  So not much worse, but KHE has one extra missing in HN_8 and one in TR_92,
  for a total extra cost of 30.  But this may trade off against unassigned
  tasks in the case of TR_92.  Perhaps the non-history laim defects are
  more worth studying.

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       56
    Y  Unnecessary assignments        19       26
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  The cost difference is (56 - 43) * 20 = 260 for avail negs, and
  (1 - 5) * 30 = 120 for unassigned tasks, making a total difference
  of 260 - 120 = 140 for these two.  The other issue is limit active
  intervals constraints, where the cost difference is 270 - 75 = 195.
  Combine these gives 140 + 195 = 335.  If we could get rid of that
  we would have a solution of cost 1590 - 335 = 1255, rel = 1.00.

  I've fiddled with HSEval so that it now prints a yellow background to
  cells containing a task printed in italics.  Much easier to find them:

                                     LOR      KHE
    ---------------------------------------------
    HN unnecessary assignments         0        4
    NU unnecessary assignments         1        3
    CT unnecessary assignments         0        2
    TR unnecessary assignments        18       17
    ---------------------------------------------
       Total                          19       26

21 August 2024.  Done the documentation and boilerplate for redundancy
  monitors, see khe_redundancy_handler.c.  Also added
  KhePreferResourcesMonitorMakeInternal which can be used to add
  redundancy monitors; it will do all the linking in etc.  In fact,
  everything is done except KheRedundancyHandlerAddMonitors and
  yourself.

22 August 2024.  Added redundancy to yourself as rrd.  All done and
  documented, including only doing it when supply <= demand.  Started
  testing, found that I was often calling KhePreferResourcesConstraintDomain
  and KhePreferResourcesMonitorDomainComplement, which don't always
  work now because there may be no constraint.  So fixed that up and
  now the testing seems to be working.  I've got 5255 lines of
  debug output, roughly one line per added monitor, all look right.

  Got right to the end, but look at the cost:

    [ "INRC2-4-100-0-1108", 1 solution, in 257.4 secs: cost 0.01684 ]

  What happened to detaching the redundancy monitors?

23 August 2024.  Fixed up all occurrences of KheMonitorConstraint.
  Ready to test (again).  Had to make sure that EnsureOfficialCost
  was ensuring that they were detached.  First results:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 279.4 secs:
      0.01625 0.01650 0.01655 0.01660 0.01685 0.01690 0.01695 0.01710
      0.01735 0.01760 0.01775 0.01845
    ] 12 distinct costs, best soln (cost 0.01625) has diversifier 9

  Slightly worse, now look at the details:

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       58
    Y  Unnecessary assignments        19       27
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

                                     LOR      KHE
    ---------------------------------------------
    HN unnecessary assignments         0        3
    NU unnecessary assignments         1        4
    CT unnecessary assignments         0        0
    TR unnecessary assignments        18       20
    ---------------------------------------------
       Total                          19       27

  This is marginally worse than before:  previously, we had 26
  unnecessary assignments.  Let's try a 10 minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.8 mins:
      0.01495 0.01565 0.01620 0.01635 0.01645 0.01650 0.01655 0.01660
      0.01670 0.01720 0.01725 0.01775
    ] 12 distinct costs, best soln (cost 0.01495) has diversifier 9

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       55
    Y  Unnecessary assignments        19       25
    X  Unassigned tasks                5        3
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  The overall score is pretty good (rel = 1.20), but the number of
  italic entries is not really better.  Here is a 10 minute run
  without rrd:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.7 mins:
      0.01565 0.01575 0.01590 0.01595 0.01600 0.01630 0.01655 0.01660
      0.01670 0.01675 0.01715 0.01725
    ]

  The 1495 score was an outlier.  But we can keep looking into it.

24 August 2024.  Here's a 10-minute run using rrd around rec only:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.8 mins:
      0.01550 0.01575 0.01575 0.01585 0.01595 0.01605 0.01615 0.01650
      0.01660 0.01670 0.01765 0.01790
    ]

  Really just a random variation.  And here is a 5-minute run where
  rrd() encloses both the first repair phase and the second:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 280.5 secs:
      0.01620 0.01625 0.01635 0.01635 0.01635 0.01655 0.01675 0.01710
      0.01745 0.01785 0.01815 0.01950
    ]

23 August 2024.  Dynamic crash, the clue is in the debug output:

    [ A1 14949 Constraint:25/TR_97 ... ]
    [ A0 16359 PreferResourcesRedundancy/3Sat:Early/A=s0+NWCaretaker=h1:1 ... ]

  The monitor is attached but it has no path to the solution object.
  Why not?

13 September 2024.  Went to PATAT2024 and returned to a week's COVID
  isolation.  On 10 September I resumed work.  I've been looking
  closely at the specification of IHTC2024, seeing whether it can
  be expressed in XESTT.  Still doing that today.

14 September 2024.  I've completed the conversion, but now I have
  to go carefully through the specification and make sure that
  every last requirement is taken care of.  Plus I need to add
  descriptions of IHTC2024 and XESTT, and a concluding discussion.

15 September 2024.  I'm close to finalizing the conversion paper.
  I need to leave it fallow for a day or two, then audit it, then
  post it to my web site, then send an email (already composed)
  to Greet and Andrea giving my comments on IHTC2024.

  Returned to the bug I found before leaving for PATAT.  I have some
  debug output now which shows that the total asst cost of the packed
  soln is 0.00001.  This could explain the discrepancy of 0.00001 in
  the total costs, if this asst cost has been added to the packed
  soln cost but is not present in the platform cost.  But why would
  it not be present in the platform cost?

  Also I've come upon this:

    ignoring PRC:NA=h1|NWTrainee=h1:1:Trainee/1Mon:Early/NA=h1|NWTrainee=h1:1
      (zero cost: rg 25, rt Nurse:Trainee 25)

  Are there really prefer resources constraints containing every
  resource?  Where did they come from?  What are they doing there?
  They are there to ensure that ordinary nurses do not get assigned
  to trainee slots.  Once we partition the solve into ordinary and
  trainee parts they are not really relevant any more.  Are they
  being mishandled, perhaps replaced by soft cost 1 or something?

16 September 2024.  Still trying to find that bug.  From what I can
  see there is no problem adding and removing asst_cost.  So what
  else could it be?

  What if asst_cost is being added to the expander cost *and* to
  the signature cost?  Wouldn't that cause it to be counted twice?
  But then, wouldn't it happen for other resources as well?  We've
  got examples of other resources with ass costs and there is no
  problem.  But it isn't being added to any signature cost, it
  is just being added to the expander cost - and taken away again.

17 September 2024.  Brought my paper on converting IHTC2024 to XESTT
  to a conclusion and sent an email about it to Andrea and Greet.

  Returning to the dynamic bug, I have some debug output that seems
  to show conclusively that the bug is inside dynamic, not imported
  from elsewhere.

18 September 2024.  Various other jobs took the whole day.

19 September 2024.  Found and fixed a bug where the increment
  in the indexed list structures was not taking asst_cost and
  non_asst_cost into account.  But sadly that did not fix the
  original bug.

  Hacked up the redundancy costs so that they are now 3 each,
  and sure enough our bug changed to be out by 3 rather than
  out by 1.  So the problems does seem to be to do with
  double counting asst_cost.  But there are many cases in
  the debug output where asst_cost is not double counted.
  The current crash on TR_98 has two redundancy monitors, but
  the discrepancy is 3 not 6.

  Tried it with expand_by_shifts=false.  There is one fewer
  addition/subtraction to de->cost, but the problem is still
  there.  So I'll go on testing that way now.

20 September 2024.  Working on the bug.  I've got quite a lot
  of debug output from the rerun, but there is no evidence
  that asst_cost is being added twice.  So I'm still stuck.

21 September 2024.  Looking into the latest theory about
  the bug.  The grouping of mtasks on 1Tue:Night is odd:

    [ time 7:
      [ MTask 1Tue
	n0.00002 a0.00000 1Tue:Night.29{}
      ]
      [ MTask 1Tue
	n0.00002 a0.00000 1Tue:Night.30{}
      ]
      [ MTask 1Tue
	n0.00000 a0.00007 1Tue:Night.31{}
	n0.00000 a0.00007 1Tue:Night.32{}
	n0.00000 a0.00007 1Tue:Night.33{}
	n0.00000 a0.00007 1Tue:Night.34{}
      ]
      [ MTask 1Tue
	n0.00000 a0.00007 1Tue:Night.35{}
      ]
    ]

  It's because TR_92 is fixed to 1Tue:Night.29, TR_88 is fixed to
  1Tue:Night.30, and TR_98 is fixed to 1Tue:Night.35.  Now we're
  getting somewhere:  the problem is that fixed tasks are not
  having their assignment costs subtracted during opening, but
  they are being added during searching.  What about monitor
  costs in general?  Are they counted twice?

  KheDrsExpanderAddTaskSoln adds dt->asst_cost and also
  calls KheDrsTaskSolnLeafSet, which is concerned with
  monitor costs.  But KheDrsTaskSolnLeafClear, the
  corresponding clear function, does not seem to be
  called during opening.  So there is a puzzle here.

  It looks like we have to distinguish between a resource on day
  being fixed and being closed.  If it's closed we don't do
  anything to change any costs, if it's merely fixed then we do
  assign it to its fixed value and update costs.

  We need fixed (with cost calculations) because it allows us to do
  reruns.  We need closed (without cost calculations) because there
  are times when we can't unassign, so we can't recalculate.
  (Can we be closed but unassigned?  At present we can only
  represent closed but assigned.)

  This needs some serious thought.  KheDrsTaskSolnLeafSet and
  KheDrsTaskSolnLeafClear specifically test for and rule out
  changes when there is a closed assignment.  Higher expressions
  are already primed, so I guess we calculate a signature even
  if the slot is closed.  What is the equivalent for asst_cost?

  Must remember to set expand_by_shifts to default false after.  Also
  to turn off the code that sets the redundancy weight to 3.  Done.

  Added a semi-intuitive fix, it seems to be working:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 277.4 secs:
      0.01565 0.01570 0.01595 0.01600 0.01640 0.01655 0.01665 0.01705
      0.01740 0.01755 0.01775 0.01800
    ] 12 distinct costs, best soln (cost 0.01565) has diversifier 11

  This seems fairly reliable, rel = 1.25.  Still a long way short.
  Roughly a week lost to this stupid bug.

22 September 2024.  Taking a close look at yesterday's 1565 solution.

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       61
    Y  Unnecessary assignments        19       29
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  The cost difference is (61 - 43) * 20 = 360 for avail negs, and
  (1 - 5) * 30 = -120 for unassigned tasks, making a total difference
  of 360 - 120 = 240 for these two.  The other issue is limit active
  intervals constraints, where history is great (almost optimal) but
  there are non-history issues leading to cost delta (165 - 75) = 90.
  If we could reduce the 1565 solution by 240 + 90 we would have a
  solution of cost 1565 - 240 - 90 = 1235, which is 10 better than LOR.

  If we could get rid of the extra limit active intervals cost we
  would have a solution of cost 1565 - 90 = 1475 which is rel = 1.18.

  If we could get rid of the extra unnecessary assignments we would
  have cost 1565 - (29 - 19) * 20 = 1565 - 200 = 1365 which would
  be rel = 1.09.  That would be everything we need.

  Swap CT_56 <--> CT_73 on 1Wed-1Fri and then unassign the unnecessary
  1Wed.  Would that work, is it being tried?  Problem is that then
  early follows late in CT_73.

  Tried with redundancy in all three phases, and got this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 278.4 secs:
      0.01530 0.01545 0.01585 0.01630 0.01640 0.01645 0.01655 0.01675
      0.01705 0.01755 0.01755 0.01770
    ] 11 distinct costs, best soln (cost 0.01530) has diversifier 11

  It's very impressive all through; rel = 1.22, and

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       60
    Y  Unnecessary assignments        19       29
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  But there is also improvement in active intervals (cost 150
  here vs 165 above).  Curiously there is no improvement in
  unnecessary assignments, even though that was the aim of
  including redundancy in all three phases.

  TR_* on 1Sat-1Sun:  LOR has 5 unnecessary assignments (or 8 counting
  from the start), KHE has 7 (or 9 counting from the start).  Fixing
  that would not change much.

  TR_* on 2Sat-2Sun:  LOR has 3 unnecessary assignments, KHE has 3.

  TR_* on 2Sat-2Sun:  LOR has 3 unnecessary assignments, KHE has 3.

  TR_* on 1Mon-4Sun, LOR has 18 unnecessary assignments, KHE has 23.
  Most of the extras are clustered around 3Wed (LOR has 1, KHE has 4)
  - why?

  Tried increasing the cost of redundancy to 20, to make it equal
  with the cost of non-assignment.  Result was

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 276.3 secs:
      0.01625 0.01650 0.01655 0.01665 0.01690 0.01695 0.01745 0.01780
      0.01825 0.01830 0.01865 0.01905
    ]

  Quite inferior, so I'm taking it away again.  Now trying a run
  of the whole archive (5 minutes only).  Got a core dump right
  near the start.  KheGroupByResourceConstraints seems to be the
  culprit.

    Instances (20) 	LOR17 		KHE24x12
   	                Cost 	Rel. 	Cost 	Rel.
    ------------------------------------------------
    INRC2-4-030-1-6291 	0.01695 1.00 	0.01825 1.08
    INRC2-4-030-1-6753 	0.01890 1.00 	0.01970 1.04
    INRC2-4-035-0-1718 	0.01425 1.00 	0.01575 1.11
    INRC2-4-035-2-8875 	0.01155 1.00 	0.01290 1.12
    INRC2-4-040-0-2061 	0.01685 1.00 	0.01850 1.10
    INRC2-4-040-2-6106 	0.01890 1.00 	0.02060 1.09
    INRC2-4-050-0-0487 	0.01505 1.00 	0.01640 1.09
    INRC2-4-050-0-7272 	0.01500 1.00 	0.01620 1.08
    INRC2-4-060-1-6115 	0.02505 1.00 	0.02835 1.13
    INRC2-4-060-1-9638 	0.02750 1.00 	0.03030 1.10
    INRC2-4-070-0-3651 	0.02435 1.00 	0.02780 1.14
    INRC2-4-070-0-4967 	0.02175 1.00 	0.02495 1.15
    INRC2-4-080-2-4333 	0.03340 1.00 	0.03620 1.08
    INRC2-4-080-2-6048 	0.03260 1.00 	0.03620 1.11
    INRC2-4-100-0-1108 	0.01245 1.00 	0.01585 1.27
    INRC2-4-100-2-0646 	0.01950 1.00 	0.02340 1.20
    INRC2-4-110-0-1428 	0.02440 1.00 	0.02675 1.10
    INRC2-4-110-0-1935 	0.02560 1.00 	0.02895 1.13
    INRC2-4-120-1-4626 	0.02170 1.00 	0.02415 1.11
    INRC2-4-120-1-5698 	0.02220 1.00 	0.02565 1.16
    ------------------------------------------------
        Average 	0.02089 1.00 	0.02334 1.12

  Last time I did this (22 May 2024), the KHE24x12 average was 2322,
  without dynamic, and 2330 with dynamic, so this is slightly worse
  than before.  But INRC2-4-100-0-1108 was 0.01650 then, so there
  has been some improvement in the worst relative cost.  Eleven of
  the 20 instances are over the 1.10 target here.  I've just found
  by hand that nine of them were over the 1.10 target on 22 May.

23 September 2024.  Grinding down INRC2-4-100-0-1108.  In the big
  run yesterday its cost was 0.01585, but I have been getting
  costs as low as 1530 on other runs.  Back to a 5-minute run
  of INRC2-4-100-0-1108 alone:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 282.3 secs:
      0.01565 0.01585 0.01585 0.01595 0.01620 0.01630 0.01680 0.01690
      0.01745 0.01755 0.01775 0.01800
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       58
    Y  Unnecessary assignments        19       29
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  This solution has rel = 1.25.

  Tried "KheSolnTryTaskUnAssignments(soln, false, options);",
  where the false value will remove unnecessary assignments.

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 278.3 secs:
      0.01530 0.01535 0.01605 0.01620 0.01630 0.01635 0.01640 0.01685
      0.01700 0.01715 0.01730 0.01790
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       60
    Y  Unnecessary assignments        19       29
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  It's different, but TR_84 3Wed is still assigned - why?  It's because
  TR_84 requires sequences of at least 3 busy days in a row.

    [ "INRC2-4-100-0-1108", 1 solution, in 273.0 secs: cost 0.01540 ]

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       27
    O  Available times (negative)     43       56
    Y  Unnecessary assignments        19       26
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

24 September 2024.  Working on KheRunHomogenize in file
  khe_sr_run_homogenize.c.  Got it running, here is a
  run with it being installed just before rec:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 276.2 secs:
      0.01580 0.01620 0.01630 0.01630 0.01655 0.01665 0.01680 0.01685
      0.01705 0.01740 0.01785 0.01800
    ] 11 distinct costs, best soln (cost 0.01580) has diversifier 6

  And here is a run without installing it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 279.3 secs:
      0.01540 0.01595 0.01600 0.01615 0.01640 0.01655 0.01660 0.01705
      0.01740 0.01755 0.01760 0.01800
    ] 12 distinct costs, best soln (cost 0.01540) has diversifier 3

  We seem to do better without it.  Here is a 10-minute run without it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.7 mins:
      0.01520 0.01540 0.01565 0.01570 0.01600 0.01605 0.01610 0.01630
      0.01645 0.01650 0.01730 0.01730
    ]

  This is rel = 1.23.  And here is a 10-minute run with it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.3 mins:
      0.01505 0.01540 0.01575 0.01580 0.01590 0.01605 0.01625 0.01625
      0.01635 0.01640 0.01655 0.01700
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       58
    Y  Unnecessary assignments        19       29
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

    Summary 					Inf. 	Obj.
    ---------------------------------------------------------
    Assign Resource Constraint (2 points) 	   	60
    Avoid Unavailable Times Constraint (6 points)    	60
    Cluster Busy Times Constraint (40 points) 	   	1190
    Limit Active Intervals Constraint (8 points)    	195
    ---------------------------------------------------------
      Grand total (56 points) 	   			1505

  This is rel = 1.20.  But U and Y seem quite bad.  Why is it
  so good in total?  Anyway, going back to a 5-minute run
  without it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 279.3 secs:
      0.01525 0.01610 0.01620 0.01635 0.01640 0.01640 0.01655 0.01690
      0.01705 0.01740 0.01755 0.01790
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       25
    O  Available times (negative)     43       52
    Y  Unnecessary assignments        19       23
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

    Summary 					Inf. 	Obj.
    ---------------------------------------------------------
    Assign Resource Constraint (1 point) 	   	30
    Avoid Unavailable Times Constraint (8 points)   	80
    Cluster Busy Times Constraint (36 points) 	   	1130
    Limit Active Intervals Constraint (13 points)    	285
    ---------------------------------------------------------
      Grand total (58 points) 	   			1525

  It looks like we need to do more work on limit active intervals
  defects.  Perhaps there is some repair that incorporates
  homogenization?

25 September 2024.  Started thinking about low-level optimization of
  ejection chains.  The code to go for is this from KheDoDecreaseLoad:

    KheForEachInterval(&swap_ii_rec, in)
      KheForEachResource(&other_ri_rec, other_r)
	KheSwapRepair(ej, ao, in, true, r, other_r, tg, true)

  and this from KheIncreaseLoadMultiRepair:

    KheForEachInterval(&swap_ii_rec, in)
      KheForEachResource(&other_ri_rec, other_r)
	if( KheSwapRepair(ej, ao, in, true, r, other_r, NULL, true) )

  Apart from the tg parameter of KheSwapRepair these two fragments are
  identical.  There is a lot of recalculation of sets of tasks for r
  and also other_r, although it's true that the sets depend on both
  resources (because they must be reassignable).  So if we change to

    KheForEachResource(&other_ri_rec, other_r)
      KheForEachInterval(&swap_ii_rec, in)
	KheSwapRepair(ej, ao, in, true, r, other_r, tg, true)

  then r and other_r are constant within the inner loop.  Could
  we do something like this:

    KheForEachResource(&other_ri_rec, other_r)
      KheForEachResourceSwapInterval(&swap_ii_rec, r, other_r, in)
	KheSwapRepair(ej, ao, in, true, r, other_r, tg, true)

  where we avoid recalculating stuff in the inner swap.  We could
  see whether the kernel interval works for r and other_r, and then
  see how far out (up to max_extra) we can go with the two of them,
  and generate all the break points at once, and then try all
  possible start break points and end break points.

26 September 2024.  Time to design MTaskSetSwapIterator.

    KheForEachResource(&other_ri_rec, other_r)
    {
      MTaskSetSwapIteratorInit(&mtask_set_swap_ii_rec, r, other_r,
        kernel_in, (other swap stuff));
      KheForEachMTaskSetSwap(&mtask_set_swap_ii_rec, mts1, mts2)
	if( KheMTaskSetSwapRepair(ej, ao, mts1, mts2) )
	  return true;
    }

27 September 2024.  Working on MTaskSetSwapIterator.  Got some
  debug output, it all seems to be working, only I still have
  to make sure that the tasks are reassignable.  All kernel
  tasks have to be reassignable, and if others are not then
  we have to reduce the bounds.

28 September 2024.  Finished KheSwapIteratorMake.  It needs an
  audit and test.

29 September 2024.  Audited all the new ejection chains code and
  made a few adjustments.  Started testing.  I began this work on
  25 September, so it has been a fairly quick job.  First results:

    [ "INRC2-4-100-0-1108", 1 solution, in 241.5 secs: cost 0.01570 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 264.9 secs:
      0.01540 0.01560 0.01570 0.01580 0.01600 0.01610 0.01625 0.01645
      0.01660 0.01660 0.01695 0.01715
    ]

  It's a pretty good result (rel = 1.23), more consistently good (4
  solutions below 1600, for example) than anything we've had before.
  But also it is running for only 4 and a half minutes, not 5.
  Altogether I think this rates as a success.

  Here we go again with es_full_widening_on=true:

    [ "INRC2-4-100-0-1108", 1 solution, in 240.1 secs: cost 0.01710 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 270.3 secs:
      0.01585 0.01585 0.01590 0.01600 0.01625 0.01640 0.01645 0.01675
      0.01680 0.01685 0.01730 0.01730
    ] 10 distinct costs, best soln (cost 0.01585) has diversifier 6

  Not so good, take it away again.  Here's a 10-minute run without it:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.4 mins:
      0.01535 0.01540 0.01545 0.01550 0.01565 0.01570 0.01580 0.01610
      0.01660 0.01660 0.01680 0.01710
    ]

  Only a marginal improvement.

30 September 2024.  Written an unassign iterator on the plan of
  the swap iterator.  Audited and ready to use.  First results:

    [ "INRC2-4-100-0-1108", 1 solution, in 241.5 secs: cost 0.01570 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 264.0 secs:
      0.01570 0.01590 0.01600 0.01605 0.01610 0.01615 0.01620 0.01660
      0.01660 0.01665 0.01690 0.01760
    ] 11 distinct costs, best soln (cost 0.01570) has diversifier 0

  Not as good, has something gone wrong?  Need to do a whynot to
  see what's being tried.

1 October 2024.  I've done the whynot but everything seems to be
  working.  So it's just chance that it has not done as well.

  Discovered that KheEventResourceMultiRepair was not using the
  swap and unassign iterators when it should have been.  So I've
  fixed that.  First results are:

    [ "INRC2-4-100-0-1108", 1 solution, in 241.6 secs: cost 0.01570 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 261.7 secs:
      0.01570 0.01570 0.01600 0.01600 0.01605 0.01605 0.01620 0.01620
      0.01655 0.01660 0.01715 0.01750
    ] 8 distinct costs, best soln (cost 0.01570) has diversifier 0

2 October 2024.  Working on KheAssignIterator.

  KheIncreaseLoadMultiRepair(r, tg).  The current algorithm is:

    for each mtask mt in tg
        (trying mtasks that need assignment before those that don't)
      if( r can be assigned to mt  )
	for each widened interval including mt
	  build a set of mtasks including mt and try assigning r to it

  KheEventResourceMultiRepair(er, ri).  The current algorithm is:

    for each task t derived from er
      if( t is unassigned )
	mt = task's mtask
	for each widened interval including mt
	  for each non-NULL resource other_r in ri
	    build a set of mtasks including mt and try assigning other_r to it

  If we change this to

    for each task t derived from er
      if( t is unassigned )
	mt = t's mtask
	for each non-NULL resource other_r in ri
	  if( other_r can be assigned to mt  )
	    for each widened interval including mt
	      build a set of mtasks including mt and try assigning other_r to it

  then the interval iterator will have a resource other_r to go on,
  and so it will be able to grab mtasks that need assignment and can
  accept that resource, like in KheIncreaseLoadMultiRepair.  So we
  can build a single set of mtasks that include mt, are assignable
  to r (or other_r), and preferably have the same offset as mt, and
  then try assigning various subsets of that set.

  Done all this and have clean compile.  Also finding the mtask
  set; I used the old code as a model.  Audited and ready to test.

    [ "INRC2-4-100-0-1108", 1 solution, in 219.6 secs: cost 0.01630 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 252.8 secs:
      0.01670 0.01880 0.01960 0.01990 0.02045 0.02050 0.02060 0.02070
      0.02105 0.02110 0.02145 0.02200
    ] 12 distinct costs, best soln (cost 0.01670) has diversifier 0

  Something's wrong, we need to do a whynot and work out what is
  not being done.  Running time is down as well.

3 October 2024.  Realized that excluding days was wrong for the
  assign iterator, because it can try multiple mtasks on one day.
  So I've removed that now.  Here are the first results:

    [ "INRC2-4-100-0-1108", 1 solution, in 233.2 secs: cost 0.01830 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 258.9 secs:
      0.01750 0.01845 0.01860 0.01945 0.01955 0.02000 0.02005 0.02020
      0.02040 0.02080 0.02085 0.02090
    ] 12 distinct costs, best soln (cost 0.01750) has diversifier 11

  Amazing, correcting the problem has made things worse.  It's the
  limit active intervals monitors, and the problems are mainly
  underloads.

6 October 2024.  Had a couple of days off doing other things.  Done
  a whynot run.  Something is very wrong with KheMTaskAssignRepair:

    [ KheMTaskAssignRepair-(TR_91, {0-7: 1Tue:Night.29, 1Tue:Night.29,
        1Tue:Night.29, 1Tue:Night.29, 1Tue:Night.29, 1Mon:Night.31,
	1Tue:Night.29, 1Tue:Night.29})
    ]

  Changed KheMTaskSetUniqueify to sort by increasing starting time
  and then increasing task index.  Done and documented.  First results:

    [ "INRC2-4-100-0-1108", 1 solution, in 228.5 secs: cost 0.01615 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 259.5 secs:
      0.01645 0.01645 0.01660 0.01670 0.01690 0.01700 0.01700 0.01715
      0.01720 0.01730 0.01740 0.01760
    ] 10 distinct costs, best soln (cost 0.01645) has diversifier 11

  Quite good but inferior to 29 September.  So I guess we go back
  to the usual grinding down.  Here is the best soln in detail:

    Summary (KHE24x12)					Inf. 	Obj.
    ----------------------------------------------------------------
    Assign Resource Constraint (1 point) 	   		  30
    Avoid Unavailable Times Constraint (7 points) 	   	  70
    Cluster Busy Times Constraint (35 points) 	   		1140
    Limit Active Intervals Constraint (18 points) 	   	 405
    ----------------------------------------------------------------
      Grand total (61 points) 	   				1645

  Comparing with LOR:

    Summary (LOR)					Inf. 	Obj.
    ----------------------------------------------------------------
    Assign Resource Constraint (5 points) 	   		 150
    Avoid Unavailable Times Constraint (7 points) 	   	  70
    Cluster Busy Times Constraint (28 points) 	   		 950
    Limit Active Intervals Constraint (3 points) 	   	  75
    ----------------------------------------------------------------
      Grand total (43 points) 	   				1245

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       54
    Y  Unnecessary assignments        19       22
    X  Unassigned tasks                5        1
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  These values are much the same as always.  It's the limit active
  intervals defects that are the new problem.  Imagine if I could
  cut them down by 300 to 105, that would be 1345, rel = 1.08.  But
  225 is a more realistic target, looking through results above.

  Trying best of 8, since I have some doubts about processor speed
  when I run 12 in parallel:

    [ "INRC2-4-100-0-1108", 8 threads, 8 solves, 243.7 secs:
      0.01645 0.01645 0.01685 0.01695 0.01715 0.01715 0.01730 0.01760
    ] 6 distinct costs, best soln (cost 0.01645) has diversifier 5

  Same best cost, others are no better.  Back to best of 12 now:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 261.7 secs:
      0.01625 0.01645 0.01645 0.01690 0.01700 0.01700 0.01715 0.01720
      0.01720 0.01730 0.01740 0.01760
    ]

  Spot of luck but still needs a lot of grinding down.

20 October 2024.  I've been away bushwalking.  Starting work today.

21 October 2024.  Thinking about where to go from here.

22 October 2024.  Starting to think about cutting my losses.
  Suppose we run for 10 minutes with wider moves?  Here is
  the result of gs_time_limit=10:00, es_move_widening_max=6,
  and es_swap_widening_max=24:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 8.3 mins:
      0.01575 0.01600 0.01610 0.01615 0.01630 0.01640 0.01680 0.01695
      0.01730 0.01775 0.01810 0.01880
    ] 12 distinct costs, best soln (cost 0.01575) has diversifier 6

  So there is some improvement there; rel = 1.26, still well short.
  Here's a 60-minute run with the same other parameters:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 36.4 mins:
      0.01530 0.01550 0.01565 0.01575 0.01590 0.01610 0.01645 0.01655
      0.01660 0.01720 0.01800 0.01835
    ] 12 distinct costs, best soln (cost 0.01530) has diversifier 4

  This has rel = 1.22.  Trying the same run with es_max_augments=500:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 37.7 mins:
      0.01530 0.01540 0.01565 0.01575 0.01590 0.01610 0.01645 0.01655
      0.01660 0.01670 0.01690 0.01800
    ]

  No improvement in the best solution, but some in the others, and
  we are still not using up the whole hour.  So I've changed to
  es_move_widening_max=8, es_swap_widening_max=32, es_max_augments=1000
  and I've reinstated rdv on the second repair phase.  This gives

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 54.2 mins:
      0.01590 0.01630 0.01635 0.01655 0.01660 0.01670 0.01740 0.01745
      0.01770 0.01790 0.01820 0.01870
    ] 12 distinct costs, best soln (cost 0.01590) has diversifier 2

  It uses up most of this time, but the result is worse.  So now
  I've returned to es_move_widening_max=6 es_swap_widening_max=28
  es_max_augments=500 but I've left the extra call to rdv in:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 55.8 mins:
      0.01620 0.01625 0.01630 0.01630 0.01630 0.01640 0.01650 0.01660
      0.01680 0.01695 0.01720 0.01725
    ]

  So now I've kept the same parameters but removed the second call to
  rdv:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 37.5 mins:
      0.01555 0.01555 0.01590 0.01595 0.01600 0.01615 0.01620 0.01630
      0.01635 0.01695 0.01705 0.01715
    ] 11 distinct costs, best soln (cost 0.01555) has diversifier 8

23 October 2024.  Back to the standard five minute run, looking for
  ways to grind it down:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 260.2 secs:
      0.01615 0.01645 0.01645 0.01690 0.01700 0.01700 0.01715 0.01720
      0.01720 0.01740 0.01740 0.01760
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       52
    Y  Unnecessary assignments        19       25
    X  Unassigned tasks                5        4
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  Here rel = 1.29.  Delta(U + Y) = (26 + 25) - (24 + 19) = 51 - 43 = 8,
  so we would expect overload cost delta about 8 * 20 = 160, when in
  fact we have 1160 - 950 = 210.  KHE is paying an extra 30 for
  weekend problems, so that explains most of the discrepancy.

  Added blue cells denoting must be busy due to history.

24 October 2024.  Working on the blue cells problem.  I've now
  verified that what is being printed is correct, but that more
  could be printed.  And now I'm printing that extra stuff, and
  I've changed the header explanation too.  All done and dusted.
  Also added red when free days are required at the start.

25 October 2024.  More thinking, mainly about the new coloured
  boxes at the start of the cycle, but no action.

26 October 2024.  Trying an experiment where I do a full-width
  swap when the sequence is too short:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 261.7 secs:
      0.01605 0.01610 0.01630 0.01640 0.01655 0.01675 0.01695 0.01710
      0.01720 0.01725 0.01750 0.01805
    ] 12 distinct costs, best soln (cost 0.01605) has diversifier 9

  It seems to have helped; the first four solutions are better than
  the previous second best.  Tried the same thing for decreasing loads:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 260.0 secs:
      0.01635 0.01685 0.01695 0.01705 0.01705 0.01750 0.01770 0.01770
      0.01775 0.01785 0.01795 0.01855
    ]

  This one is worse.  Is it just random?  Going back to previous.

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 262.0 secs:
      0.01625 0.01645 0.01645 0.01650 0.01690 0.01700 0.01700 0.01715
      0.01720 0.01720 0.01730 0.01760
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       51
    Y  Unnecessary assignments        19       24
    X  Unassigned tasks                5        4
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  I need something with a clearer signature of improvement.  The
  U + Y numbers are so small that it's hard to see any improvement
  in them.  Chances might be better with limit active intervals
  defects, where the KHE cost is 255 but LOR is only 75, a difference
  of 180.  If that was removed we would have rel = 1.16.

  CT_53 has history 7 days off which is the max, so it must
  be busy on the first day.  It needs a blue box there.  Done.

  Removed double swaps, just to see if they are still useful:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 260.1 secs:
      0.01625 0.01630 0.01635 0.01650 0.01675 0.01680 0.01695 0.01730
      0.01740 0.01750 0.01780 0.01795
    ]

  Hmmm, no improvement in the best value.  Very marginal.  Anyway
  I'm keeping them.

  Maybe the way around these barriers is more solutions.  Here is
  a run in which 24 solutions each ran for 10-minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 24 solves, 16.0 mins:
      0.01545 0.01600 0.01600 0.01610 0.01635 0.01645 0.01645 0.01650
      0.01655 0.01660 0.01665 0.01670 0.01680 0.01695 0.01695 0.01700
      0.01705 0.01705 0.01705 0.01725 0.01750 0.01760 0.01770 0.01800
    ]

  And here is a run in which 48 solutions each ran for 20 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 48 solves, 54.4 mins:
      0.01495 0.01530 0.01535 0.01560 0.01565 0.01575 0.01590 0.01595
      0.01600 0.01600 0.01605 0.01610 0.01610 0.01615 0.01620 0.01625
      0.01630 0.01635 0.01635 0.01645 0.01650 0.01655 0.01655 0.01655
      0.01660 0.01665 0.01670 0.01670 0.01670 0.01670 0.01680 0.01685
      0.01685 0.01695 0.01695 0.01700 0.01705 0.01735 0.01750 0.01750
      0.01760 0.01760 0.01760 0.01765 0.01770 0.01780 0.01790 0.01805
    ] 35 distinct costs, best soln (cost 0.01495) has diversifier 43

  It's slow.  The result, 1495, is rel = 1.20.

27 October 2024.  Here is a run in which 48 solutions each ran for
  five minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 48 solves, 17.6 mins:
      0.01560 0.01565 0.01610 0.01620 0.01620 0.01625 0.01625 0.01630
      0.01640 0.01640 0.01645 0.01645 0.01650 0.01655 0.01660 0.01665
      0.01670 0.01675 0.01680 0.01690 0.01690 0.01690 0.01700 0.01700
      0.01705 0.01705 0.01710 0.01715 0.01720 0.01720 0.01720 0.01725
      0.01730 0.01735 0.01745 0.01750 0.01755 0.01755 0.01760 0.01765
      0.01795 0.01800 0.01815 0.01825 0.01830 0.01840 0.01840 0.01840
    ] 35 distinct costs, best soln (cost 0.01560) has diversifier 27

  And here is a run in which 12 solutions each ran for 20 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 15.0 mins:
      0.01535 0.01575 0.01590 0.01605 0.01610 0.01625 0.01645 0.01645
      0.01670 0.01695 0.01695 0.01760
    ] 10 distinct costs, best soln (cost 0.01535) has diversifier 9

  Generally speaking it seems best to run each solve for the full
  available time i.e. always run 12 solves.  Here is a run of 12
  solves for 60 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 35.3 mins:
      0.01535 0.01575 0.01580 0.01600 0.01615 0.01630 0.01635 0.01645
      0.01670 0.01695 0.01695 0.01705
    ] 11 distinct costs, best soln (cost 0.01535) has diversifier 9

  Now let's return to the standard 12 runs for 5 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 263.7 secs:
      0.01645 0.01645 0.01645 0.01660 0.01700 0.01700 0.01700 0.01715
      0.01715 0.01730 0.01740 0.01760
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       53
    Y  Unnecessary assignments        19       24
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

28 October 2024.  Finished off KheMoveUnnecessaryAssignments in
  file khe_sr_unnecessary.c.  Now it needs an audit and test.
  First results:
  
    [ KheMoveUnnecessaryAssignments(soln, Nurse, options)
      KheMoveUnnecessary (NU_17, HN_12): 3Sun:Early.4 <--> {3Sun:Early.3}
      KheMoveUnnecessary (CT_58, CT_39): 3Sat:Day.14 <--> {3Sat:Day.10}
      KheMoveUnnecessary (CT_61, CT_45): 3Sun:Early.12 <--> {3Sun:Early.8}
    ] KheMoveUnnecessary returning
    [ KheMoveUnnecessary(soln, Nurse:Trainee, options)
      KheMoveUnnecessary (TR_78, TR_94): 3Sat:Early.25 <--> {3Sat:Early.24}
      KheMoveUnnecessary (TR_78, TR_94): 3Sun:Early.21 <--> {3Sun:Early.20}
      KheMoveUnnecessary (TR_80, TR_94): 3Sat:Early.26 <--> {3Sat:Early.25}
      KheMoveUnnecessary (TR_80, TR_94): 3Sun:Early.22 <--> {3Sun:Early.21}
    ] KheMoveUnnecessary returning
    [ KheMoveUnnecessary(soln, Nurse, options)
      KheMoveUnnecessary (HN_7, HN_2): 3Sun:Early.1 <--> {3Sun:Early.0}
      KheMoveUnnecessary (CT_61, NU_17): 3Sat:Day.14 <--> {3Sat:Day.12}
    ] KheMoveUnnecessary returning
    [ KheMoveUnnecessary(soln, Nurse:Trainee, options)
      KheMoveUnnecessary (TR_80, TR_94): 3Sat:Early.25 <--> {3Sat:Early.26}
      KheMoveUnnecessary (TR_80, TR_94): 3Sun:Early.21 <--> {3Sun:Early.22}
      KheMoveUnnecessary (TR_81, TR_77): 1Wed:Early.31 <--> {1Wed:Early.32}
      KheMoveUnnecessary (TR_84, TR_92): 1Sun:Early.20 <--> {1Sun:Early.19}
      KheMoveUnnecessary (TR_90, TR_89): 1Wed:Day.36 <--> {1Wed:Day.34}
      KheMoveUnnecessary (TR_91, TR_75): 4Tue:Day.38 <--> {4Tue:Day.36}
      KheMoveUnnecessary (TR_96, TR_92): 1Sun:Early.21 <--> {1Sun:Early.20}
      KheMoveUnnecessary (TR_99, TR_77): 1Wed:Early.30 <--> {1Wed:Early.31}
    ]

  These look good.  The usual five-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 252.0 secs:
      0.01605 0.01615 0.01650 0.01665 0.01690 0.01715 0.01715 0.01720
      0.01720 0.01750 0.01790 0.01875
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       52
    Y  Unnecessary assignments        19       25
    X  Unassigned tasks                5        4
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  This does seem somewhat better; the first two solutions are better
  than the previous best.  But on the other hand, the number of
  unnecessary assignments is one larger than before.  Looking at
  the table, most of the unnecessary assignments do seem to be
  for overloaded resources, so that's something.  Here's another run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 250.5 secs:
      0.01615 0.01650 0.01660 0.01665 0.01690 0.01715 0.01715 0.01715
      0.01720 0.01720 0.01750 0.01790
    ]
                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       26
    O  Available times (negative)     43       51
    Y  Unnecessary assignments        19       22
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  This has rel = 1.29.  The extra workload is (26 + 22) - (24 + 19) =
  48 - 43 = 5, so the extra cost should be about 100.  In fact
  it is 190, why is that?   Weekends stuff is KHE 120, LOR 90,
  so that explains 30.  Then the 3 extra assignments are 60 so
  that explains the rest.

29 October 2024.  Looking at TR_88, it has two history entries, one
  with weight 15 saying that its last timetable ended with 1 night
  shift, so that it needs at least 3 night shifts now; and the other
  with weight 30 saying that its last timetable ended with 4 busy
  days, so that it needs at most 1 busy day now.  The LOR solution
  assigns it one night shift, which seems like the best option on
  the whole.  Why didn't the KHE solution do that too?  Because it
  requires an unnecessary night assignment?
  
    Resource     LOR        KHE
    ------------------------------------------------
    HN_8           1          2
    TR_88          2          3
    TR_92                     1
    TR_98          2          2
    TR_78                     1 (too many free days)
    ------------------------------------------------

  Two of these, for TR_88 and TR_92, are explained by LOR
  going for two unnecessary assignments to fill these gaps.
  So this just leaves HN_8 (no clear reason) and TR_78 (too
  many free days).

  TR_78 is not assigned at all by assign by history.  Which makes
  sense.  It is assigned by KheDynamicResourceSequentialSolve, in
  fact it is number 2 on the list.  And now I have some debug
  output which shows that KheDynamicResourceSequentialSolve
  assigns tasks to TR_78 on 1Tue and 1Wed.  So these must have
  been taken away by a repair later.

  HN_8 is assigned by KheAssignByHistory to tasks on 1Mon, 1Tue, 
  and 1Wed, just as it should be.  So the 1Tue and 1Wed assignments
  must have been deleted by later repairs.

30 October 2024.  Here is the current resource solver:

    "gts!do(rem("
      "1: rt(rrd(rin!do(RBW(rwp(rcm(RAH(rgc(rcx!RDS)))))))),"
      "2: rt(rrd(rin!do(rgc(rrm, rfs, rmu, rec)))),"
      "1: rt(red, rrm, rdv, rmu, rec, rrm)"
    "))"

    "gts!do(rem(rt(rin!do
    )

  rrd is redundancy handling, adding a prefer resources monitor
  of small cost to penalize unnecessary assignments.

  rwp is KheWorkloadPack. the special case code for the instance
  where everything has workload 7 or 10.

  rcm is KheClusterMinimumSolverMake, for adding minimum limits
  derived from maximum limits.

  rgc is KheGroupByResourceConstraints.

  I need to audit it wrt task fixing and make a plan.

  Implementing new do-it-yourself syntax.  I have a clean compile,
  it needs an audit and tidy up.

31 October 2024.  Auditing the revised do-it-yourself syntax.  I've
  revised the documentation.  It now specifies that some solvers
  are enclosing and others are non-enclosing.  I've implemented that.

1 November 2024.  Audited the documentation and made the implementation
  agree with it.  I've tested it and adjusted the debug output and it
  all seems to be working well.

  Updated KheBalanceWeekends and its documentation so that it works
  properly with do-it-yourself solving.  This involved removing the
  no_undo option, basically.  If you want no undo you can ensure
  that rbw applies all the way to the end.

  Updated KheAssignByHistory and its documentation so that it
  works properly with do-it-yourself solving.  It now fixes its
  assignments when the sa parameter is non-NULL.  Do-it-yourself
  solving always passes a non-NULL sa, so it always fixes.

  I've decided to do nothing with KheDynamicResourceSequentialSolve.

  ----------------------solvers that fix tasks------------------
  rbw: KheBalanceWeekends either fixes all optional tasks on the less
  busy day of a weekend, or else it fixes a selection of required
  tasks on the busier day of a weekend.  The fixes can either be
  undone later (using a solution adjuster), or not, as determined
  by the rs_balance_weekends_no_undo option.  So it's a bit messy here.

  rah: KheAssignByHistory fixes the assignments it makes.  In fact it
  offers three choices:  don't fix; fix using a solution adjuster
  so that the fixes can be undone later; and fix without using a
  solution adjuster so that the fixes are permanent.  These choices
  are determined by the rs_assign_by_history_fix option, whose
  default value gives no fixing.  But at present we are fixing.

  rds: KheDynamicResourceSequentialSolve optionally fixes the optimal
  assignments it makes (controlled by option rs_drs_fix, whose
  default value is false).

  KheTaskingDoAvoidSplitAssignmentsConstraintJob for task trees
  fixes the assignments it makes which avoid split assignments.

  KheFindSplitResourceAssignments unfixes the assignments made
  by KheTaskingDoAvoidSplitAssignmentsConstraintJob.  So does
  KheTaskingAllowSplitAssignments.
  ----------------------solvers that fix tasks------------------

  First test of revised solver:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01585 0.01655 0.01690 0.01730 0.01770 0.01775 0.01780 0.01790
      0.01795 0.01805 0.01825 0.01920
    ] 12 distinct costs, best soln (cost 0.01585) has diversifier 9

  The best result is pretty good, but after that it drops off
  quite quickly.  I think there is some fiddling around with
  fixing to be done here.  Rel = 1.27.

2 November 2024.  I've fiddled with the default value of the "rs"
  option and got this from the standard 5-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01575 0.01640 0.01645 0.01650 0.01695 0.01710 0.01715 0.01725
      0.01730 0.01745 0.01745 0.01935
    ] 11 distinct costs, best soln (cost 0.01575) has diversifier 7

  It's similar to yesterday; the best result is good, but the gap
  between best and second best is disturbingly large.  Still the
  second best is pretty good too.  I'll carry on from here.
  Running again with more time for ejection chains:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01640 0.01645 0.01650 0.01690 0.01705 0.01705 0.01715
      0.01730 0.01730 0.01745 0.01930
    ] 10 distinct costs, best soln (cost 0.01565) has diversifier 7

  Corresponding solutions are always as good or better, but is there
  enough in it to justify the extra complexity?  Tried again with rdv
  turned off altogether:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 240.0 secs:
      0.01570 0.01665 0.01670 0.01670 0.01675 0.01695 0.01695 0.01700
      0.01730 0.01735 0.01760 0.01870
    ]

  These are basically random variations.  Back to equal time:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01595 0.01640 0.01645 0.01695 0.01705 0.01705 0.01710 0.01715
      0.01730 0.01740 0.01745 0.01935
    ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01575 0.01645 0.01650 0.01675 0.01675 0.01705 0.01710 0.01725
      0.01730 0.01745 0.01780 0.01935
    ]

  I guess the question now is whether any of the construction
  refinements are worth adding to the first repair phase.  If
  we add rrd to the first repair phase, we get this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01595 0.01635 0.01665 0.01680 0.01725 0.01730 0.01735 0.01750
      0.01760 0.01785 0.01830 0.01925
    ]

  The best is worse, but the second best is better.  Adding rcm:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01575 0.01640 0.01645 0.01650 0.01690 0.01705 0.01710 0.01720
      0.01725 0.01725 0.01775 0.01935
    ]

  Adding rbw:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01595 0.01640 0.01645 0.01650 0.01695 0.01705 0.01710 0.01725
      0.01730 0.01740 0.01775 0.01935
    ]

  Deleting rgc:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01690 0.01695 0.01715 0.01745 0.01790 0.01805 0.01835
      0.01840 0.01880 0.01940 0.01950
    ]

  The best is surprisingly good, but the second best is quite bad.
  What these tests seem to show is that the current default value
  of "rs" is as good as anything.  Now what about removing each
  refinement from the construction phase?  Removing rrd:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01725 0.01725 0.01730 0.01735 0.01745 0.01760 0.01800
      0.01840 0.01885 0.01925 0.01950
    ]

  Removing rcm:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01575 0.01640 0.01645 0.01675 0.01715 0.01715 0.01715 0.01725
      0.01730 0.01740 0.01745 0.01935
    ]

  Removing rbw:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01660 0.01705 0.01715 0.01745 0.01760 0.01775 0.01785 0.01790
      0.01820 0.01830 0.01905 0.01940
    ]

  That's one that has really made a difference.  Removing rah:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01655 0.01675 0.01685 0.01710 0.01715 0.01715 0.01780 0.01790
      0.01795 0.01880 0.01885 0.01910
    ]

  Another one that really matters.  Back once more to the standard:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01580 0.01625 0.01640 0.01650 0.01675 0.01705 0.01705 0.01710
      0.01730 0.01745 0.01760 0.01955
    ]

  Here rel = 1.26.  And here is a 60-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01550 0.01615 0.01630 0.01630 0.01630 0.01635 0.01635 0.01650
      0.01665 0.01680 0.01700 0.01835
    ]

  Not a lot of improvement over the 5-minute run.  OK, it's time
  now to go back to grinding down the 5-minute result, which is:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01580 0.01640 0.01645 0.01650 0.01675 0.01705 0.01705 0.01710
      0.01730 0.01760 0.01775 0.01955
    ]

    Summary 						LOR     KHE
    ---------------------------------------------------------------
    Assign Resource Constraint (5 points) 	   	150      60
    Avoid Unavailable Times Constraint (7 points)    	 70      70
    Cluster Busy Times Constraint (28 points) 	   	950    1240
    Limit Active Intervals Constraint (3 points) 	 75     210
    ---------------------------------------------------------------
      Grand total (43 points) 	   		       1245    1580

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       56
    Y  Unnecessary assignments        19       25
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

    LOR:  950 - (60 + 30) =       860 / 20 = 43 O defects.
    KHE: 1240 - (30 + 30 + 60) = 1120 / 20 = 56 O defects.
    Difference in U + Y is (28 + 25 ) - (24 + 19) = 53 - 43 = 10.

  Saving those 10 would give rel = (1580 - 200)/ 1245 = 1.10.
  At present we have rel = 1580 / 1245 = 1.26.

    Limit active intervals defects       LOR     KHE
    ------------------------------------------------
    History defects                       75     120
    Non-history defects                    0      90
    ------------------------------------------------
                                          75     210

  We've verified previously that assign by history does very
  well, and that it is just subsequent repairs that spoil it.
  Here's a run in which the assign by history assignments
  remain fixed until the end of the run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01685 0.01755 0.01785 0.01815 0.01815 0.01830 0.01830 0.01855
      0.01860 0.01865 0.01875 0.01910
    ]

    Limit active intervals defects       LOR     KHE
    ------------------------------------------------
    History defects                       75      90
    Non-history defects                    0     135
    ------------------------------------------------
                                          75     225

3 November 2024.  Added busy days constraints to assign by
  history.  All documented, implemented, and tested.  It
  seems to be working but a careful audit would be good.
  First results:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01700 0.01705 0.01705 0.01715 0.01760 0.01775 0.01785 0.01830
      0.01845 0.01850 0.01900 0.01990
    ]

  These are very poor results!  Why?  Because we are fixing
  history assignments so we can see how they turn out.  And
  indeed they are turning out very well, the 1700 solution
  above has just two history defects, total cost 45.  Now
  freeing up those assignments, we get this:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01645 0.01700 0.01740 0.01740 0.01745 0.01755 0.01755 0.01780
      0.01810 0.01845 0.01865 0.01890
    ]

    Summary 						LOR     KHE
    ---------------------------------------------------------------
    Assign Resource Constraint (5 points) 	   	150     120
    Avoid Unavailable Times Constraint (7 points)    	 70      80
    Cluster Busy Times Constraint (28 points) 	   	950    1130
    Limit Active Intervals Constraint (3 points) 	 75     315
    ---------------------------------------------------------------
      Grand total (43 points) 	   		       1245    1645

  It's not a great result, I need to look into why not.  If I could
  get the same limit active result that I had before (210) I would
  save 105 and the total would be 1540, which would be a good result.
  Here's a run without rrd:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01650 0.01690 0.01700 0.01725 0.01745 0.01765 0.01770 0.01790
      0.01795 0.01820 0.01855 0.01900
    ]

  It's marginally worse.  Here's another run with rrd, same as
  before, but magically we've got back to a good best result:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.1 mins:
      0.01560 0.01695 0.01700 0.01705 0.01745 0.01745 0.01760 0.01775
      0.01785 0.01810 0.01870 0.01885
    ]

  This is better than every 5-minute run from 1 November.  Here's
  another go, a 60-minute run this time:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 60.0 mins:
      0.01540 0.01620 0.01625 0.01665 0.01685 0.01695 0.01725 0.01740
      0.01745 0.01755 0.01765 0.01770
    ]

  This is slightly better than the 1550 I got from the previous
  60-minute run, on 2 November.  Back to the 5-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.1 mins:
      0.01560 0.01700 0.01710 0.01740 0.01745 0.01755 0.01755 0.01810
      0.01830 0.01845 0.01855 0.01895
    ]

  The best result is good (rel = 1.25), but the second best is way
  behind.  It would be good to have more consistency in these results.

4 November 2024.  Yesterday's last result (rel = 1.25), in detail:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.1 mins:
      0.01560 0.01700 0.01710 0.01740 0.01745 0.01755 0.01755 0.01810
      0.01830 0.01845 0.01855 0.01895
    ]

                                     LOR      KHE
    ---------------------------------------------
    U  Available times (positive)     24       28
    O  Available times (negative)     43       54
    Y  Unnecessary assignments        19       23
    X  Unassigned tasks                5        2
    ---------------------------------------------
    U - O + Y - X                     -5       -5

  Delta(U + Y) = (28 + 23) - (24 + 19) = 51 - 43 = 8.  So would
  expect Delta(AR + CB) around 160, whereas in fact

    Summary 						LOR     KHE
    ---------------------------------------------------------------
    Assign Resource Constraint (5 points) 	   	150      60
    Avoid Unavailable Times Constraint (7 points)    	 70      60
    Cluster Busy Times Constraint (28 points) 	   	950    1170
    Limit Active Intervals Constraint (3 points) 	 75     270
    ---------------------------------------------------------------
      Grand total (43 points) 	   		       1245    1560

  Delta(AR + CB) = (60 + 1170) - (150 + 950) = 1230 - 1100 = 130.  This
  is close to the 10% result we are looking for: 130 / 1100 = 0.11.
  Or if we include the first three lines we get

    (60 + 60 + 1170) / (150 + 70 + 950) = 1290 / 1170 = 1.102

  The problem is with limit active intervals defects:

    Limit active intervals defects       LOR     KHE
    ------------------------------------------------
    History defects                       75     105
    Non-history defects                    0     165
    ------------------------------------------------
                                          75     270

  We're a long way off target here.  We need to save 0.15 * 1245 = 190
  in order to get to rel = 1.10, that is, reduce these costs to 80.

  Whynot swap TR_77 with TR_94 + TR_95 on 1Mon-1Wed?  Because TR_94
  has 5 free days of history so it must be busy on 1Wed or earlier.

5 November 2024.  Starting work on mtask grouping today.  Documentation
  is done, also boilerplate.  When adding tasks we do this:

    KheMTaskFinderAddTask
      build sig;
      KheMTaskFinderFindMTask (makes mts, finds mt or NULL)
	KheMTaskSetAddMTask(mts, mt) (mtasks_at_time or mtasks_at_meet)
	HaArrayAddLast(mtf->mtasks) (used for traversing all mtasks)
      KheMTaskAddTask
	add task to mt
	KheMTaskFixAssignments (sort out order within mt)
	KheMTaskFinderLinkTaskToMTask (sets mtf->mtasks_by_task)

    I need clean code to add a task and delete a task from the
    structure.  At present it does seem to be a bit convoluted.
    And I need a structural submodule for mtasks and a structural
    submodule for mtask finders.  I need to separate out the
    structural code so that I can call it from other functions.

6 November 2024.  Working on adding task grouping to mtask finding.
  So far I've updated the documentation and made sure that khe_solvers.h
  agrees with it.  I've also reorganized khe_sr_mtask_finder.c to
  agree with the documentation and khe_solvers.h.

7 November 2024.  Getting somewhere with khe_sr_mtask_finder.c.  I now
  have private functions KheMTaskFinderAddTask and KheMTaskFinderDeleteTask
  which I can use to add and delete a task from the mtasks at any time.
  I've implemented all the forbidden operations except the grouping
  operations.

8 November 2024.  Audited yesterday's stuff and finished off the
  forbidden operations.  So what's next?  Replacing KHE_TASKER_CLASS
  by KHE_MTASK.

  Adjusted soln adjuster, it can now move tasks and it works when
  sa == NULL.

  Created file khe_sr_comb_grouper.c and started work on it.  My
  plan is to change the names and replace the profile requirements
  by the callback function requirements, then take stock.
	  
9 November 2024.  Working through khe_sr_comb_grouper.c.  Have
  clean compile but quite a lot has been left out.
 
10 November 2024.  Working through khe_sr_comb_grouper.c.  Written
  some documentation saying that assigned tasks are ignored by
  grouping by resource constraints.

  Working through khe_sr_comb_grouper.c from start to finish.  I've
  just finished KheMTaskCovers (line 638 of 2080).
 
11 November 2024.  Finished working through khe_sr_comb_grouper.c.
  I've started on the documentation now.  Found a couple of good
  simplifications of the code.  Working on comb grouper documentation.
  All done but needs an audit.
 
12 November 2024.  Working on comb grouper documentation.

  Confusion over KheCombGrouperAddMTaskInfo vs KheMTaskGroupingAddMTask.
  Do we need both functions?  Yes, for data abstraction, because one
  deals with mti, the other with mt.

  Reviewed element time set.  During initialization it gets set for
  time groups but not for mtasks, what is that about?  It's about
  calculating coverage, it's OK as is.

  Changed how costs are worked out.  Everything is based now on the
  interval of days that the mtasks included in the solver are running.
 
13 November 2024.  Auditing khe_sr_comb_grouper.c.  Did a fairly large
  rewrite to bring the recursive generation of all subsets of the
  mtasks to a more conventional structure, one that recursively
  tries each mtask in turn, first including it, then excluding it.
  The old method was more complex, and even supposing that it worked
  it was not easy to be sure that it worked.
 
14 November 2024.  Audited khe_sr_comb_grouper.c, documentation and
  code, hopefully for the last time.  All good.
 
15 November 2024.  Done KheFindCombGrouping.  Pretty easy, just
  changing some names and types.  Also its documentation is done.

  Started work on KheFindProfileGrouping.  I've updated the
  documentation, although it hasn't really changed.  I've read
  it all taken it in.  Now for khe_sr_profile_grouper.c.
 
16 November 2024.   Now for khe_sr_profile_grouper.c.  I've worked
  through it and got something that is compilable, but what it
  does I don't really know.  Time to work through it and sort
  out what it does and what to do about the bits that I've had
  to comment out.

  Removed pos_groups and distributed it over the profile time groups.
  I've verified that the code is good from line 908 onwards (functions
  KheProfileGroupingForMonitors, KheFindProfileGrouping,
  KheGroupForResourceType, and KheGroupByResourceConstraints).
 
17 November 2024.   Working on khe_sr_profile_grouper.c.

18 November 2024.   Working on khe_sr_profile_grouper.c.  Have
  new data structure planned out, and started work on it.  I've
  just commented out KheMTaskFinderMTaskAtTimeCount and
  KheMTaskFinderMTaskAtTime (and documented this), so I'm
  currently up to line 4466.

19 November 2024.   Working on khe_sr_profile_grouper.c.  Have
  clean compile after changes to khe_sr_mtask_finder.c.

21 November 2024.   Still working on khe_sr_mtask_finder.c.
  Documented and implemented a plan from yesterday that makes
  degenerate tasks and mtasks not a special case, and also gets
  rid of accessing mtasks via their meets.  Deleted a lot of
  commented-out code.  What's left is now 4035 lines.

22 November 2024.   Final audit of khe_sr_mtask_finder.c.
  Now have a separate table for each resource type.  Removed
  the etm parameter, it was not used.

  KheEntryMake - why is there no KheEntryDelete?  There
  is a free list.  I guess I might need KheEntryDelete
  in the future, but I don't need it now.  Once a time
  group or interval is requested it stays on the books.

  KheMTaskFinderRetrieveSig is done with a three-state sig
  type approach now, much better.  But needs an audit; am
  I really finding the mtasks with similar signatures?

  Added debug output.

  I've deleted the etm parameter from quite a lot of functions.
  I really need to go through the documentation and see who is
  claiming to need etm when they really don't.  Could I even
  not use the etm option at all?  No, I've checked everything
  and you can run "grep 'etm =' khe*.c" to get a list of all
  the files that actually use etm.  There are still quite a lot.

23 November 2024.  I've written an implementation note section
  of the Guide explaining the data structures and purporting to
  show why an entry for a particlular time group or interval
  needs to be added to the list for every time of that time
  group or interval.  It needs an audit but it looks good.

24 November 2024.  Audited khe_sr_mtask_finder.c against the
  new section of the documentation.  All good.

  Reviewed khe_sr_comb_grouper.c (it accepts requirements
  and solves a comb problem) and khe_sr_comb_finder.c (it
  implements KheFindCombGrouping, which is largely about
  combination elimination but also tries combinatorial
  grouping over various intervals).  It seems all right.

  Started work on khe_sr_profile_grouper.c.  I've changed it
  to group limit active intervals monitors with the same
  time groups.

25 November 2024.  Working on khe_sr_profile_grouper.c.  I've
  audited the documentation and added a few paragraphs on what to
  do about tasks with very different assignment costs.  I've also
  added KheCombGrouperAddPreferredDomainRequirement to the comb
  grouper, documented it, and used it in the profile grouping code.
  But the main thing is, I've finished khe_sr_profile_grouper.c.

  Removed the old grouping by resource constraints documentation.
  Updated the makefile.  I need to remove 3 source files and
  comment out their interfaces in khe_solvers.h.

26 November 2024.  Finished tidying up, including passing a
  solution adjuster around.  All done and documented.  Removed
  the two callback functions from KHE_COMB_GROUPER, since
  it seems they are not needed after all.

  Started testing, found the first bug, no doubt of many.

  If there is a cover requirement with no last cover, we
  can already abandon the search.  I've just added that in.

27 November 2024.  I've documented the fact that you could ask
  to cover an mtask which is fixed or fully assigned.  This
  condition cannot be satisfied, but you can ask for it.

  And I've changed khe_sr_comb_finder.c so that it does not
  try solves that attempt to cover fixed or fully assigned
  tasks.

  Got a clean run:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01870 ]

  And again:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01930 ]

  I've been working on mtask grouping since 5 November 2024, which
  is three weeks.  Why has it taken so long?

  The groups I'm getting are not quite right.  Needs looking into.

28 November 2024.  Revising KheCombinedResourceAssign.  It was wrong
  because it was calling grouping twice.  The second call, at the
  start of repair, made no sense because nearly every task that
  matters has been assigned by then.

  The first line of "rs" contains these items:

      rin!do    Apply resource invariant
      rrd       Redundancy (withdraws weight changes afterwards)
      rwp       Workload pack (withdraws domain changes afterwards)
      rcm       Cluster Minimum (withdraws min limits afterwards)
      rbw       Balance weekends (withdraws asst fixes afterwards)
      rah       Assign by history (withdraws asst fixes afterwards)
      rgc       Group by constraints (withdraws groups afterwards)

  This all looks good.  I'm now using this value for rs:

    gts!do rem rt
    (
      3: rin!do rrd rwp rcm rbw rah rgc
      (
	  1: rcx!rts,
	  2: (rfs, rrm, rec, rdv)
      ),
      1: (red, rrm, rec, rdv)
    )

  so that rgc is only called once.  So far the results are not good:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.02005 ]

  But at least we are getting the right calls to rgc now.  We now
  need to make sure we are getting the right groups.  There are
  probably still too many combinatorial groups and too few (in
  fact no) profile groups.

  Added an optimization to the comb grouper that kills off mtasks
  that clash with required mtasks before the solve begins.

  The reason why we don't have any grouping of senior nurses is
  that they can also take nurse roles, so there are two zero-cost
  groupings:

    {1.00000: |1Sat:Night.0|, 1Sun:Early.0}
    {1.00000: |1Sat:Night.0|, 1Sun:Early.5}
    {1.00000: |1Sat:Night.0|, 1Sun:Day.0}
    {1.00000: |1Sat:Night.0|, 1Sun:Day.5}
    {1.00000: |1Sat:Night.0|, 1Sun:Late.1}
    {1.00000: |1Sat:Night.0|, 1Sun:Late.5}
    {0.00000: |1Sat:Night.0|, 1Sun:Night.0}
    {0.00000: |1Sat:Night.0|, 1Sun:Night.5}

  But for caretakers the choices are:

    {1.00000: |1Sat:Night.10|, 1Sun:Early.10}
    {1.00000: |1Sat:Night.10|, 1Sun:Day.10}
    {1.00000: |1Sat:Night.10|, 1Sun:Late.10}
    {0.00000: |1Sat:Night.10|, 1Sun:Night.10}
    {0.00030: |1Sat:Night.10|}

  In other words, caretakers can only take the caretaker role.
  These solves seem to be doing the right things and getting
  the right results.  We may get more grouping when we get
  profile grouping working.

  Found and fixed a bug in profile grouping, then got this:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.03585 ]

  Something is very wrong but I can only keep hacking along.

29 November 2024.  Now requiring all tasks that participate in grouping
  to need assignment.  I've documented this, and implemented it for
  comb grouping and profile grouping, ready to test.  I have also
  added KheMTaskNeedsAssignmentCount to KHE_MTASK (documented and
  implemented).  Did a 5-minute test, got this:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.02330 ]

  Not a good result, something must still be wrong.  I've had a
  quick look but at present I don't know what the problem is.

  Some of the groups ended on a Saturday.  That can't be good.  e.g.

    {0.00000: |4Tue:Early.15{4Wed:Early.13, 4Thu:Early.15, 4Fri:Early.14}|,
      4Sat:Early.12}

  The problem is that the code was taking the union of the days that
  the selected mtasks are running, which is not enough.  The old code
  took the union of the days that the requirements were running.  So
  now I am extending busy_in by one day on each side.  This has
  indeed got rid of profile groups that end on a Saturday.  However
  the solution it found is still very poor:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.02425 ]

  So more testing to do.

  I've verified that only tasks that need assignment are being
  grouped.  As for domains:

    {0.00000: |3Thu:Early.6|, 3Fri:Early.7}
    Domains are Nurse and Nurse.

    {0.00000: |4Mon:Early.17|, 4Tue:Early.16{4Wed:Early.14, 4Thu:Early.16}}
    Domains are Caretaker, Caretaker, Caretaker, and Caretaker.

    {0.00000: |4Mon:Day.0{4Tue:Day.0}|, 4Wed:Day.7}
    Domains are HeadNurse, HeadNurse, and Nurse.

    {0.00000: 4Tue:Day.5, |4Wed:Day.0|}
    Domains are Nurse and HeadNurse.

  So they seem a bit wobbly but not too bad.

30 November 2024.  Working on a more carefully structured traversal
  of the domains at each time group.  Added type KHE_PROFILE_DOMAIN
  and boilerplate functions for it, but no new algorithmic code yet.

  1Mon:Early need assignment count: 8
    HeadNurse   1
    Nurse       1
    NWCaretaker 7

  1Tue:Early need assignment count: 11
    HeadNurse   1
    Nurse       2
    NWCaretaker 8

  On Mon I count 9 but it says 8, this suggests that one task
  is already assigned (by assign by history), presumably a
  Caretaker because that would allow us to start two groups
  of Caretaker on 1Tue, which is what is done:

    forward profile grouping: 1 x {0.00000: |1Tue:Early.5|, 1Wed:Early.5}
    forward profile grouping: 2 x {0.00000: |1Tue:Early.12|, 1Wed:Early.14}

  I've looked over the groups being made, both by combinatorial
  grouping and by profile groupiing, and they look pretty darn
  good.  So why is the result so bad?

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.02445 ]

  Best of 12:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.02445 0.02465 0.02545 0.02605 0.02610 0.02625 0.02655 0.02725
      0.02790 0.02915 0.02925 0.02935
    ] 12 distinct costs, best soln (cost 0.02445) has diversifier 3

  These are seriously hopeless solutions.  What's going wrong?

  I've realized that just moving the task back again in soln adjuster
  is not right.  What if it was assigned a resource in the meanwhile?
  We need to move it to whatever it was grouped with.  This is
  what task sets used to do - they knew it was a group/ungroup.
  So I've added a KheSolnAdjusterTaskGroup function and I'm now
  calling it when grouping tasks.  Here are the first results:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01795 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01725 0.01740 0.01740 0.01780 0.01810 0.01815 0.01825 0.01840
      0.01860 0.01865 0.01920 0.02005
    ]

  It's a lot better but we are still a long way above what we
  were getting before (e.g. 0.01560 on 4 November 2024).  So
  I need to keep slogging.

  I looked over the 1725 result and there is no obvious problem
  with it.  So now I'm trying a 10 minute run just to see where
  we end up:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 10.0 mins:
      0.01635 0.01665 0.01710 0.01730 0.01765 0.01770 0.01780 0.01815
      0.01820 0.01825 0.01875 0.01915
    ]

  Maybe on the old runs I was not getting grouping on the
  first repair phase.  We grouped twice but the second time
  around it may not have worked because so much is assigned.
  Now I am getting real grouping on the first repair phase.
  So we'll now test a 5-minute run, but with rgc only on
  construction, not on the first repair phase:

    gts!do rem rt
    (
      1: rin!do rrd rwp rcm rbw rah rgc rcx!rts,
      2:        rrd rwp rcm rbw         (rfs, rrm, rec, rdv),
      1: (red, rrm, rec, rdv)
    )

  The result of this is:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01575 0.01595 0.01610 0.01645 0.01650 0.01675 0.01675 0.01685
      0.01715 0.01735 0.01755 0.01820
    ]

  OK, so now we are back in the ballpark of the 1560 result we were
  getting before.  Interesting.  Here's a 10-minute run with the same rs:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 10.0 mins:
      0.01515 0.01535 0.01550 0.01570 0.01570 0.01590 0.01615 0.01625
      0.01630 0.01635 0.01645 0.01655
    ] 11 distinct costs, best soln (cost 0.01515) has diversifier 10

  Gosh, is this a new best?  I think it might be, although there was
  a 1495 (best of 48 runs) some time back.  1515 is Rel = 1.21.  And
  the runners-up are pretty good too.  Let's try 30 minutes:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01485 0.01510 0.01530 0.01535 0.01555 0.01570 0.01570 0.01610
      0.01615 0.01620 0.01620 0.01645
    ]

  OK, now this really is a new best.  Rel = 1.19.  Here's a 5 minute
  run with no grouping:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01640 0.01660 0.01675 0.01675 0.01690 0.01780 0.01795 0.01825
      0.01830 0.01845 0.01850 0.02000
    ]

  Not a bad result, but still it's inferior, and look at the
  worst result, it's quite bad.  Back to the usual 5-minute
  run, got this this time:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01595 0.01645 0.01650 0.01675 0.01675 0.01690 0.01700 0.01710
      0.01720 0.01735 0.01820 0.01840
    ]

  Actually I think we could cut to the chase now:  Constraint 17.
  In the 1595 solution, limit active intervals defects cost 315,
  as opposed to 75 in LOR.  So if we can get down to the LOR value
  we will have cost 1595 - (315 - 75) = 1595 - 240 = 1355 which has
  rel = 1.08.  So for the first time there is a real pathway to 1.10.

  Tried with a repeat of the second repair phase, like this:

    gts!do rem rt
    (
      1: rin!do rrd rwp rcm rbw rah rgc rcx!rts,
      2:        rrd rwp rcm rbw         (rfs, rrm, rec, rdv),
      1: (red, rrm, rec, rdv)
      1: (rrm, rec, rdv)
    )

  The result of a 5 minute run was

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01615 0.01630 0.01655 0.01685 0.01720 0.01755 0.01760
      0.01775 0.01775 0.01830 0.01875
    ]

  This does look to be generally better.  Try it again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01570 0.01620 0.01670 0.01675 0.01685 0.01685 0.01710
      0.01715 0.01755 0.01765 0.01900
    ]

  And again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01565 0.01620 0.01630 0.01695 0.01705 0.01715 0.01720 0.01730
      0.01740 0.01765 0.01775 0.01885
    ]

  OK, I think I'll stick with this for now.

1 December 2024.  Tried sorting domains into increasing size order:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01570 0.01645 0.01670 0.01680 0.01725 0.01735 0.01740 0.01745
      0.01810 0.01815 0.01820 0.01835
    ]

  And again:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01540 0.01615 0.01660 0.01690 0.01720 0.01745 0.01765 0.01800
      0.01800 0.01835 0.01850 0.01880
    ]

  I can't really see an argument that sorting matters, but I'll
  keep doing it for now.  Cost 1540 is rel = 1.24.

  Started thinking about conecutive night shift grouping.  Even
  started writing a new subsection of the grouping section.  I've
  now got what looks like a complete, workable dynamic programming
  algorithm for this problem.

2 December 2024.  Audited the section describing the dynamic programming
  algorithm for profile grouping.  It's ready to implement.

  I've reorganized the task grouping by resource constraints code
  into a subsystem with its own header file (khe_sr_tgrc.h) and
  several source files (khe_sr_tgrc_*.c).  All sorted with a clean
  compile.

3 December 2024.  Wrote a paragraph on the complexity of the problem.
  Implemented KheFindNewBest in khe_sr_tgrc_profile.c, with the
  consequential changes to khe_sr_tgrc_comb.c.  Also added fixing
  of leader tasks to KheMTaskGroupingExecute, and used it to finish
  off one round by executing its mtask groups.  So all done except
  for actually calling KheProfileTryDynamic.

4 December 2024.  Working on the overall organization of the
  TGRC code.  I've added a free list of KHE_MTASK_GROUPING
  objects, and made sure that they all go on the list and
  never leak.  I've also tidied up the calculation of cost
  for mtask groupings.  And hidden away type KHE_EC_SOLVER.

  Made sure that monitors whose time groups have fewer times
  come before monitors whose time groups have more times.

5 December 2024.  Still working on profile grouping.  Calling
  KheProfileSolverMakeGroups now instead of KheMTaskGroupingExecute;
  it calls KheMTaskGroupingExecute but it also marks the profile
  time groups out of date.

6 December 2024.  Worked through the documentation, I have
  several sections on the algorithms, all done, and two
  sections on the implementation.  All documented now.
  Also realized that the dynamic programming algorithm will
  work on any subsequence whose profile elements are positive.

7 December 2024.  Audited the code today - all done.  Changed
  KHE_MTASK_GROUPING to KHE_MTASK_GROUP.

8 December 2024.  Updated the dynamic code to solve an arbitrary
  non-zero subsequence.

9 December 2024.  Spent the day pondering using dynamic programming
  to solve the general profile grouping problem.  I can't say that
  I got very far.  Also audited the dynamic code.

  Started testing.  I've run into a mysterious bug, seems like
  an arena memory fault but that is hardly possible surely.

10 December 2024.  Fixed the bug.  I was assuming that an array
  was initialized to one entry for each time, but it was empty.
  First results:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01650 ]

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 5.0 mins:
      0.01555 0.01590 0.01675 0.01725 0.01785 0.01800 0.01805 0.01815
      0.01820 0.01845 0.01845 0.01855
    ]

  Not brilliant but at least it's working and in the ballpark.
  Here's a 30-minute run:

    [ "INRC2-4-100-0-1108", 12 threads, 12 solves, 30.0 mins:
      0.01510 0.01535 0.01555 0.01555 0.01560 0.01590 0.01595 0.01620
      0.01630 0.01635 0.01635 0.01725
    ] 10 distinct costs, best soln (cost 0.01510) has diversifier 1

  Not far off the recent new best (1485 on 30 November).  I need to
  look carefully at what it's producing.

11 December 2024.  I've been fiddling with the printing and comparing
  it with a conventional print.  It's all good now but there are still
  too many ungrouped tasks.

12 December 2024.  Switching track to Krystallidis' bug report today.

13 December 2024.  Working on Krystallidis' bug report today.  The
  non-terminating thing was about bin packing one partition into
  another.  I merely checked sizes first before starting the main
  bin pack, and that was enough, although I have also checked the
  case where the largest element of the smaller partition is 1.

  Finished getting Krystallidis' instances to work today, and
  started putting together a new version (2.13) to post asap.
  The bugs were:

    * Bug in mtask finder, to do with fixed times or not, easily fixed.

    * In khe_partition.c, a bin pack was taking forever.  I added a
      couple of trivial shortcuts and now it is not a problem.

    * KheDrsSoftIndexedSolnSetAddSoln: soft_cost % increment != 0.
      This problem afflicts the initial cost, so it arises from
      having monitors not included in the initial cost which have
      smaller cost than the increment.  I fixed it by including
      the initial cost in the increment calculation.

14 December 2024.  Put Version 3.13 out today.  And started thinking
  about a real dynamic programming algorithm for profile grouping.

15 December 2024.  Made a tentative start on profile grouping using
  dynamic programming.  Made a file (khe_sr_tgrc_dynamic_profile.c)
  and added some types and a dominance function, just to test the
  waters.

16 December 2024.  Working on khe_sr_tgrc_dynamic_profile.c.  Written
  everything except KheExtendSoln.  663 lines so far.

17 December 2024.  Working on khe_sr_tgrc_dynamic_profile.c.  Audited
  what's done so far, and added KheDpgSolverSequenceCost to find costs.
  All good and ready to start on KheExtendSoln.  I've written some
  helper code for KheExtendSoln, notably type KHE_DPG_EXTENDER.

18 December 2024.  Working on khe_sr_tgrc_dynamic_profile.c.  I've
  got something written for everything except the base of the
  extension recursion.  However, I'm currently not calculating
  any costs at all, and I'm not sure whether I am correctly
  incorporating the case where a new task decides to be the
  first in its group.  Currently I seem to not be using the
  symmetry breaking provided by the prev parameter for that.

19 December 2024.  Working on khe_sr_tgrc_dynamic_profile.c.
  1243 lines.  Clean compile and nothing "still to do".  But
  I don't really believe in it at the moment.

20 December 2024.  Worked on the documentation today.  It's
  getting closer to being something I can write code off.

21 December 2024.  Worked on the documentation today.  I've
  managed to calculate an upper bound on tbe number of
  distinct solutions for each time group.  It is polynomial
  in the number of tasks running during that time group,
  and exponential in the upper limit.  Altogether I feel
  much more in control of things now, ready to work through
  the implementation and get it right this time.

22 December 2024.  Reworking the implementation, it's going
  pretty well.  In fact it is probably finished.

23 December 2024.  Finished a straight through audit.  Now I
  need to do a functional audit, then I'll be ready to test.
  Currently 1522 lines.  I started this on 14 December.

24 December 2024.  Audited the KheDpgExtenderExtend again.
  I've made quite a few changes.  Started testing, found
  one silly bug (forgetting that prev_ds could be NULL),
  but now it's running and the debug output is looking
  rather overwhelming.

25 December 2024.  Merry Christmas to anyone foolish enough
  to be readng this diary.  Have to do some serious thinking
  about what debug output to generate, and then generate it.
  Seems to be working in some sense but it is running very
  slowly.  Number of solutions is 1, then 16, then 180, the
  1200, then gone quiet.  Is dominance testing working?

  I'm now throwing away all but the best 100 solutions on
  each day.  It's now getting to the end, but still not fast
  enough for a structural solver.  And looking at the groups,
  they are not great.  The first few days are good, so I'm
  guessing that it is the truncating to 100 that is causing
  the problems.

  I'm getting some reasonable-looking output now, but I need
  to optimize the algorithm so that I don't throw so much
  away.  And then I need a way to introduce a small number
  of optional tasks to be used as a last resort.

  Over two million solutions made in one case:

    DpgTimeGroupSolns(4Tue4, 2126388 made, 2040 undominated)

  But only 2040 undominated.  Can we do something?

26 December 2024.  Working on optimizing the dynamic programming
  algorithm.  Fixed another bug (actually the same one, using
  KheTaskDuration where KheTaskTotalDuration was wanted) and
  now I'm getting a run time of 122 seconds, when the cutoff
  is 100.  So still too slow.

27 December 2024.  Sorted out yesterday's problem with a patch.
  Today's problem is that despite a cutoff of 100 solutions,
  the running time is still 57 seconds.  Lots of solutions:

    [ KheProfileGroupingByDynamicProgramming(Constraint:17/HN_0, 4-5)
      [ DpgTimeGroupSolns(1Mon4, 1 made, 1 undominated) ]
      [ DpgTimeGroupSolns(1Tue4, 128 made, 7 undominated) ]
      [ DpgTimeGroupSolns(1Wed4, 1344 made, 84 undominated) ]
      [ DpgTimeGroupSolns(1Thu4, 43008 made, 504 undominated) ]
      [ DpgTimeGroupSolns(1Fri4, 335088 made, 3602 undominated) ]
      [ DpgTimeGroupSolns(1Sat4, 7643 made, 46 undominated) ]
      [ DpgTimeGroupSolns(1Sun4, 244 made, 118 undominated) ]
      [ DpgTimeGroupSolns(2Mon4, 3344 made, 222 undominated) ]
      [ DpgTimeGroupSolns(2Tue4, 100208 made, 1118 undominated) ]
      [ DpgTimeGroupSolns(2Wed4, 110144 made, 1350 undominated) ]
      [ DpgTimeGroupSolns(2Thu4, 56912 made, 784 undominated) ]
      [ DpgTimeGroupSolns(2Fri4, 394350 made, 5469 undominated) ]
      [ DpgTimeGroupSolns(2Sat4, 5584 made, 67 undominated) ]
      [ DpgTimeGroupSolns(2Sun4, 278 made, 142 undominated) ]
      [ DpgTimeGroupSolns(3Mon4, 2248 made, 185 undominated) ]
      [ DpgTimeGroupSolns(3Tue4, 12568 made, 389 undominated) ]
      [ DpgTimeGroupSolns(3Wed4, 22592 made, 1020 undominated) ]
      [ DpgTimeGroupSolns(3Thu4, 171328 made, 1738 undominated) ]
      [ DpgTimeGroupSolns(3Fri4, 693868 made, 4801 undominated) ]
      [ DpgTimeGroupSolns(3Sat4, 2000 made, 12 undominated) ]
      [ DpgTimeGroupSolns(3Sun4, 84 made, 48 undominated) ]
      [ DpgTimeGroupSolns(4Mon4, 2048 made, 240 undominated) ]
      [ DpgTimeGroupSolns(4Tue4, 338304 made, 1139 undominated) ]
      [ DpgTimeGroupSolns(4Wed4, 112664 made, 1154 undominated) ]
      [ DpgTimeGroupSolns(4Thu4, 57952 made, 581 undominated) ]
      [ DpgTimeGroupSolns(4Fri4, 69819 made, 2273 undominated) ]
      [ DpgTimeGroupSolns(4Sat4, 13702 made, 128 undominated) ]
      [ DpgTimeGroupSolns(4Sun4, 776 made, 1 undominated) ]
    ] KheProfileGroupingByDynamicProgramming ret. 56 (56.5 secs)

  Can this be right?  For example, 3Fri4 makes 693868 solutions,
  which means that each of the 100 solutions we started from
  makes on average 6938.68 solutions.  It seems like a lot.

  Code for timetable printing all done and running.  There seem to
  be too many solutions on 1Tue4.  They are solutions that leave a
  lot of singleton groups behind, more than I thought we were leaving.
  So this needs some careful thought.

28 December 2024.  I've documented what I hope is the answer to the
  problem of excluding undersized groups where possible.  Implemented
  and testing now.  First result:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01705 ]

  but more important is the run time (MAX_KEEP=100):

    KheProfileGroupingByDynamicProgramming ret. 56 (0.8 secs)
    KheProfileGroupingByDynamicProgramming ret. 14 (0.0 secs)

  This is for the two resource types Nurse and Nurse:Trainee.
  now again for MAX_KEEP=500:

    KheProfileGroupingByDynamicProgramming ret. 56 (1.8 secs)
    KheProfileGroupingByDynamicProgramming ret. 14 (0.0 secs)

  There is DpgTimeGroupSolns(2Fri4, 37050 made, 1988 undominated),
  which suggests that having a reasonable MAX_KEEP is important.

  I'm allowing assigned to group with unassigned, of course,
  but this is a mistake when there is another assigned to
  handle.  There does not seem to be a clever way to handle
  this, so I will just have to hack through some code that
  finds these tasks and groups them at the start of the run.
  All done but not yet tested.

  Adding assigned tasks to the mix, all done except I need
  to audit the code for increasing the group length when
  we are at the start and there is history.  Otherwise
  ready to test.

29 December 2024.  Made a horrible patch for assigned tasks that
  have fixed assignments.  Got history going, it's good now.
  Also now taking account of history and group assignments in
  dominance checking.  Not a great result, all the same:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01830 ]

  Enhanced the cost cache to accept negative first_index values.

  Trainees are getting 0 solutions again.  It goes wrong on 1Tue.

30 December 2024.  Looking into why Trainees are getting 0
  solutions again.  The cost cache is empty, suggesting that
  no-one has tried to find out the cost of the task group
  that is ending (the one assigned TR_88).

31 December 2024.  Now have a new method of finding group
  costs that does not call the combinatorial grouper at all.
  Fixed some bugs but now it seems to be working:

    [ "INRC2-4-100-0-1108", 1 solution, in 5.0 mins: cost 0.01605 ]

  Not a bad cost actually; but still I need to rethink how
  profile grouping, as now redone, fits into things generally.
  

To Do
=====

  ------------- task grouping by resource constraints -----------------
  What about this.  Run the new algorithm once and get the best soln.
  For each case where there is an undersized group, look for optional
  tasks that could extend it.  Then run again with those optional
  tasks included in the solve.

  Parameter fix_leaders_sa will become redundant if we withdraw the
  old dynamic programming algorithm.  In fact there is a lot of
  not-wonderful stuff that could be withdrawn eventually if the
  new dynamic programming algorithm is able to take over all the work.

  Things to do in profile grouping (or not):

    * I am really only profile grouping Constraint:17 at the moment.
      I was profile grouping other constraints before.  But do I
      need profile grouping of other constraints?

    * There are points where the best thing to do would be to
      incorporate a task such that asst_cost >= non_asst_cost,
      but at present there is no way to do that.  Alternatively,
      we could leave this for later repair to discover.  But if
      they are there at the start, they offer things that could
      be globally useful during dynamic programming.  What about
      a repair at the end of the expansion?

    * When dynamic programming is working, think about removing
      file khe_sr_tgrc_profile.c, moving the small amount of still
      useful code from there to khe_sr_tgrc_dynamic_profile.c.
      Then have a single dpg solver running multiple solves.

  Implement the new dynamic programming algorithm for profile grouping.
  Two issues:

     * We need to minimize domain sizes or something to keep things
       as consistent as possible, domain-wise.  Perhaps if we pass
       in an empty domain, that should handle the smallest domains
       first.

  We can prefer a domain but what about insisting on a domain,
  or say on a particular mtask that has that domain?  Surely
  we need this during profile grouping?  And domains might
  help to reduce the number of zero-cost groupings?  But we
  are already allowing any number of them, and sorting by
  domain is likely to do the job just as well.

  The main problem is limit active intervals defects, and by far
  the main problem with them is Constraint 17, which limits the
  number of consecutive night shifts to 4 or 5.  So perhaps we
  need a task grouper which handles the case of limits K and K+1.
  It would need to take history into account, which we can do
  by running it after assign by history and omitting assigned tasks
  from the grouping.

  * Assign by history is probably fine as is.  But look into it.
  ------------- task grouping by resource constraints -----------------

  Off-site backup when the revised task grouping by resource constraints
  is working.

  Change optimal assignment by dynamic programming so that when
  there is just one resource it handles grouped tasks optimally,
  by calculating a signature all the way to the end of the group
  and inserting into the table for that day.  But wait until the
  rest is done, so that it can be tested easily.

  Task bound groups could be and should be replaced by soln
  adjusters.

  Back to the usual grinding down.

  The Late-Late-Night-Night-Night-Night thing in LOR CT_58 is
  clever, it saves 30 points.  Can I build it when grouping?

  If swapping to the end helped, what about swapping back to
  the front?  There is a history problem, but anyway.

  What about some more non-trivial repairs?  Let's look for them.

  Looking at limit active intervals defects.  Both solutions have 3
  history defects; KHE is paying 30 + 45 + 15 = 90 for them, LOR is
  paying 15 + 30 + 30 = 75.  Not a big difference.  The real problem
  is that KHE is paying a further 255 - 90 = 165 for non-history limit
  active intervals defects, whereas LOR is paying 0 for them.  If I
  could get rid of the 165 excess here, that would be 1625 - 165 = 1460
  which is rel = 1.17.

  Looking at cluster busy times defects.  Both solutions pay 90
  for defects with weekends, and otherwise the defects are all
  workload overloads.  The difference in the number of these
  defects is (1150 - 950) / 20 = 10.  This roughly agrees with
  U + Y which for KHE is 28 + 22 = 50 and for LOR is 24 + 19 = 43,
  giving a difference of 7.

  Grinding down INRC2-4-100-0-1108.  It's not just limit active
  intervals defects, every kind of defect needs looking into.

  3Wed seems to be the epicentre of unnecessary assignments.
  But TR_84 and TR_85 have unnecessary assignments then because
  they require their busy days to come in groups of 3 or more.

  Can we merge KHE_TASKER_CLASS into KHE_MTASK?  Both represent a
  set of tasks running at the same times.  KHE_TASKER also has a
  type KHE_TASKER_TIME, but I've verified that I could make that
  private and just use KHE_TIME instead outside khe_sr_tasker.c.
  The only real ring-in is profile_times.  It would need a rethink.
  I would also need to merge KHE_TASKER into KHE_MTASK_FINDER, or
  just get KHE_TASKER to use mtasks instead of task classes.

  Let's get back to the main game, which is cluster and limit
  active intervals repairs.

  ------------- Fun facts about INRC2-4-100-0-1108 --------------------
  Times:  4 weeks, 4 times per day
  Nurses:  75 nurses + 25 trainees = 100 nurses

  There are hard assign resource constraints, and soft ones with
  weight 30.  There are many hard preferences for head nurses etc.
  There is a moderate number of avoid unavailable times constraints;
  their weight is only 10.  There are hard unwanted patterns:
  no non-Night shift after a Night shift, no Early or Day after
  a Late shift.  Complete weekends, weight 30.

                  Busy    Weekends   C-Free C-Working
          Weight:  20         30        30     30
    -------------------------------------------------
    FullTime     14-20       0-2       2-3    3-5
    PartTime      7-15       0-2       3-4    4-7
    HalfTime      7-10       0-1       3-7    3-5
    20Percent     3-6        0-1       2-7    2-5
    -------------------------------------------------

  Limits on consecutive same shift days are present and are the same
  for all resources:  Early 2-5, Day 2-28, Late 2-5, Night 4-5, all
  weight 15.  Unavailable times have weight 10.

  There are four grades:  HeadNurse, Nurse, Caretaker, and Trainee.
  Trainee tasks and nurses are quite separate and form a separate
  instance.  Every HeadNurse is also a Nurse (but not a Caretaker);
  every Nurse (who is not a HeadNurse) is also a Caretaker.
  ------------- Fun facts about INRC2-4-100-0-1108 --------------------

  What about this for removing italic entries.  Swap with someone
  who is overloaded, so that the italic entry is now in the
  timetable of the overloaded resource.  Then unassign the
  italic entry and possibly a neighbouring entry.  We remove
  the overload, which will pay for the unassignment if we
  reduce the overload by 2.

  e.g. HN_2 why not unassign the italic Late and its neighbour?

  Can we right-justify the numeric columns in HSEval?

  KheIncreaseLoadDoubleSwapMultiRepair and KheDoubleSwapRepair
  seem to be working.  Can I use KheDoubleSwapRepair elsewhere?
  What about something clever for fixing unavailable times defects?
  Or the elephant: workload overloads?

  What about profile grouping of night shifts, given that they
  have to occur in blocks of 4 or 5?  I've just read over the
  doc for that and there is a lot of relevant stuff there.
  And it says that the interaction with history is not implemented,
  which could explain the troubles we are having at the start
  of the timetable.  So there is a whole programme of work here.
  The first step is to find out what grouping we are getting now.

  Three-resource repairs for too-short sequences.  Get one
  fragment from one resource, another from another, and
  put them together into a longer sequence that goes to a
  third resource.  Or conversely, take a short sequence,
  break it into two pieces, and send each piece to a different
  resource.  These are probably best kept for level 1.

     r1 delete too-short sequence S1
     r2 delete too-short sequence (or part of longer sequence) S2
     r3 accepts S1+S2.

  or (probably more promising)

     r1 delete too-short sequence S1+S2
     r2 add S1 where r2 is free
     r3 add S2 where r3 is free

  But if S1+S2 is too short, S1 and S2 will be extra short.
  They need to add on where they make existing sequences longer.
  How does all of this fit with expanding?  Not wanted, I guess.
  Level 1, no expansion, only when free but adjacent to a busy time.
  The trouble is, after looking at the defects actually present
  in the current solution, none of them would be fixed by this.

  How's this?  (1) Run time sweep (2) identify all cases of
  positive limit active intervals that are too short; (3)
  unassign them (4) identify ways to group the unassigned
  intervals with other intervals that gives the right lengths;
  (5) do the grouping then unassign; (6) carry on.  It's
  fairly radical but it might just work.  Another option
  would be some form of profile grouping at the start.

  What about a full-width swap, but stopping and testing as
  we go to see if we have a better (or at least continuable)
  state?

  What about a repair that breaks a run of length 2 into two 
  runs and reassigns them to different resources?  There are
  a lot of options, we'd have to be careful to limit the
  choices, e.g. to suitable resources that are free at those
  times.

  Ejection chains:  "free" or "freer", "busy" or "busier"

  Do some detailed analysis of INRC2-8-030-1-27093606.xml and the
  other INRC2-8 instances.  The results I'm getting are quite poor;
  they should be better.

  (High school timetabling)  Take a look at UK-SP-06 from XHSTT-2014.
  The results are quite a lot worse than they used to be.

  (High school timetabling)  Is there a repair that swaps the times
  of two meets that share a preassigned resource?  There ought to be,
  and it ought to work pretty darn well.

  (High school timetabling)  There are places when KheSwapRepair and
  KheMoveRepair can't be called because there is no mtask finder
  during time repair.  I need to be able to execute these kinds of
  repairs not just on mtasks but also on tasks - or something.

  In INRC2-8-030-1-27093606 there are mtasks that do not satisfy the
  usual conditions (no gaps etc.).  I need to look into these mtasks.
  At present DRS just returns, after doing nothing, in these cases.

  Would it make sense to use the diversifier to set es_swap_widening_max?
  I've rarely done that kind of thing before.

  "if the domain allows unassignment only, try a double move"
  I understand that the domain could be empty, which means that
  unassignment is the only possibility, but what use is a double
  move in that case?  I guess we unassign the task and then
  assign the resource being unassigned to some other task at
  that time.  Yes, it does make sense - can we implement it?

  I've been looking into where KheTaskFinderMake is called from.
  There is a call from KheSolnTryTaskUnAssignments, but there
  does not seem to be any pressing need to use a task finder
  there.  It has been done so that sequences of adjacent tasks
  can be unassigned together.  But the key operation,
  KheFindTasksInInterval, could be implemented in the resource
  timetable module.  This leaves

    khe_se_solvers.c (now deleted)
    khe_sr_reassign.c
    khe_sr_single_resource.c (I may delete this module)

  where there are calls to KheTaskFinderMake.

  Appendix dynamic_impl.sig.correlators still to write.

  The existing task finder needs gs_event_timetable_monitor, which we
  can only reasonably create when time assignment is all finished.  So
  we seem to have already ruled out time adjustments during resource
  adjustments.  MTasks are not necessarily as bad as that, but we
  will need to see if we can avoid gs_event_timetable_monitor when
  fixed_times is false.  This would mean that we could only move
  tasks that are currently assigned resources and thus can be
  accessed from those resources' timetables.

  --------- An interesting variant of ejection chains -------
  Here's an interesting proposal for a variant of ejection chains.
  Have just one repair operation, which is a minimum-cost bipartite
  matching of all resources to their tasks, over an arbitrary
  sequence of adjacent days.  Each resource can match with the
  sequence of tasks initially assigned to itself or to some
  other resource, or alternatively to a free day.

  When building the bipartite graph, we may choose to leave
  out certain edges.  For example, if we are trying to unassign
  a resource at a certain time, we leave out all the edges that
  assign it to some task.  Or if we are trying to assign it
  then, we leave out its current edge, the one that causes
  it to be unassigned.  Actually we could just nominate the
  resource or task and declare that its current edge should
  be omitted.

  Then we link together these rematchings using ejection
  chains.  We decide on a repair of a single task or set of
  tasks, which is to either force their assignment or force
  their unassignment, and we make this happen by building
  the bipartite graph with the appropriate omissions.

  How to prevent cycling within one chain is a question.
  We could use the usual "don't visit the same monitor
  twice" or rather "don't alter the same monitor twice".
  Alternatively, we could insist that no chain visit any
  given day twice.

  The existing rematch module allows you to build the
  demand nodes for a given set of supply nodes and re-use
  those demand nodes.  I could use that; I could cache
  sets of demand nodes as they are built for the first time,
  and re-use the cached values.  The actual solve call
  would need to be passed a fixed non-edge separately.

  (1) Decide to move some task, or sequence of adjacent tasks.  We
      need good analysis to produce a smallest set of adjacent
      tasks that is likely to work well for the current defect.

  (2) Weighted bipartite match over that sequence of adjacent days.

  Before building the whole matching, see the effect of the
  proposed change on the resource or task affected, and only
  proceed if it improves the initial defect and does not
  introduce any more.  Then build the full graph and proceed.
  Could bury this whole aspect into the repairer.

  One repair equals one time interval plus a (supply, demand)
  edge such that that edge must be omitted from the graph.
  This will drive the solution away from the current solution.
  --------- An interesting variant of ejection chains -------

  (The stuff below here refers to the dynamic programming algorithm.)
  What about an A* search, not for pruning but for choosing
  the next solution to expand?  The trouble is it would not
  prove anything, unless the A* estimate was known to be a
  lower bound on the actual cost.  To get that we would
  basically have to do an optimal assignment of one resource
  from the current point to the end.  Too slow, surely.

  Can we run the algorithm on multiple cores?  If we do this
  successfully we could reduce the running time by a factor
  of 12, or say 10.  But how?  Perhaps we need to lock each
  index within the indexed list data structure.  Nasty.

  At "Here is the code (omitted above) to build", the shift
  solution trie section moves from trie construction to a form
  of expansion by resources.  This latter part probably
  belongs elsewhere.

  If solutions did have a common parent type including a
  signature, we could unify code that adds a solution to
  a solution set, doing dominance testing along the way;
  although the code to free a solution object would need
  a type switch.

  Do a review of signature caching.  It never seemed to speed
  anything up, although it should have done, and the code for
  it may have decayed.

  A merge of KHE_DRS_SIGNATURE and KHE_DRS_SIGNATURE_SET might be
  good.  It would be a tagged union of the two types, basically.
  The point is that then users don't have to worry about how
  signatures are put together.  A signer could have the same
  merged structure, and it might be able to build structured
  signatures just as it builds unstructured ones now.  Perhaps
  not even tagged, perhaps one part holds states and another
  part holds sub-signatures (but not sub-sub-signatures?).
  Another way to merge them would be to make KHE_DRS_SIGNATURE
  private to KHE_DRS_SIGNATURE_SET, so that everything is a
  signature set, and users of KHE_DRS_SIGNATURE now would
  have to use a signature set containing one signature.

  What about hashing the key first?  If it strikes an exact match
  we get a definite answer immediately:  the one with smaller cost
  gets deleted.  But what are the chances?  Not good, I think, but
  I am not sure.  Actually a trie would be faster and more definite.
  For assign by resources we could drop down the tree as we build.
  But if the new solution replaces the old, we still have to do
  the full thing.

  A must-assign task must get assigned to someone.  Can we use
  that to predict poor performance on the next day?  Or is there
  any other way to uncover correlation between, as opposed to
  within, resources?

  I need to focus on the first two days of solving for five trainees
  (and indeed four, although for four the problem does not really hit
  until making 1Fri).  Here's what I wrote on 19 February 2023:

    "Like before, I had to abandon 5 trainees:

      [ KheDrsSolveSearch(5 resources, 14 days)
	KheDrsSolveSearch ending day 1Mon (made 3041, undominated 3041)
	KheDrsSolveSearch ending day 1Tue (made 433770, undominated 34418)
	... (killed before getting this far)
      ]

    The number 3041 is reasonable, as the following argument shows.
    Each trainee has a choice of 4 shifts plus a free day, making 5
    choices altogether.  (Because there are excess slots we can say
    that in practice all 4 shifts are available to all trainees.)
    So there are about 5 * 5 * 5 * 5 * 5 = 3125 choices.  And on
    subsequent days, for each undominated solution on the previous
    day there are about 3125 choices, although some of them will be
    killed off very early by hard constraints, which explains why
    we do not generate anything near 3041 * 3125 day 2 solutions."

  This is the basic remaining problem.  Compared with the number
  of solutions that could be made on Day 2, the number actually
  made is small:  433770 / (3041 * 3125) = 0.05.  And compared
  with the number made, the number of undominated solutions kept is
  also small:  34418 / 433770 = 0.08.  But despite these positives
  the algorithm is being overwhelmed by large numbers of solutions.

  Sorting by weight shaved about 20% off the run time, and
  visiting the hard constraint entries before the soft ones
  when dominance testing shaved off another 30% (amazing),
  down to 6.4 seconds.  So anything we can do to speed up
  one dom test will be well worth doing.  Any other ideas?

  Solver seems to be working now, but still it is not fast enough
  to reassign five resources, or indeed four trainees.  I need
  another good idea.

  I recently moved "included_free_resources_index + 1" to what I
  thought was a better location, but now that I see it documented
  I am much less sure.  Look at it again.

  Did breaking up resource expand begin into two stages actually
  achieve anything?  KheDrsExpanderOpenToExtraCost is called but
  does not seem to be affected by the breakup.

  EvalSignature:  where is it presented, do the calls to it
  make sense to the reader, e.g. in KheDrsAsstToShiftMake?

  Make the solver return early (with failure) if time runs out.

  Signature value adjustment may need a rethink for sequence
  monitors.

  May need to revisit the current plan of always returning when
  the available cost goes negative.  This is because sequence
  monitors can contribute a positive amount to available cost.
  At any rate we should do some testing to see which is faster.

  Good idea:  compare old dom test with new dom test, and
  if there are cases where the old test succeeds and the
  new one does not, look into it.  Also vice versa.

  ==== dynamic programming ideas above this point, general ideas below ====

  If we want to combine ejection chains with dynamic programming,
  it might actually be easier to add ejection chain code to the
  dynamic programming module.  Limit task sets to the ones that
  the resources were freed from.  Could do that now, actually.

  What about a solver that swaps around the assigned shifts,
  without assigning or unassigning any resource, with the
  aim of getting the number of consecutive same shifts right.
  Is that a tractable problem?  Surely ejection chains do that?

  Look into how the resource assignment invariant interacts
  with the new "rs" option.

  It seems to be time to do some serious testing of the VLSN solver
  and compare what we get with what Legrain got.  My paper says he
  got 1695, his paper (tt/patat21/dynamic_papers/legrain.pdf) says
  1685.  My own best result from my own paper was 1835.

  Legrain's running time is m x 6n + 60, where m is the number of
  weeks and n is the number of nurses.  For the test instance,
  m = 4 and n = 30, so this is 4 x 6 x 30 + 60 = 13 minutes.

  Testing ./doit in tt/nurse/solve.

  Reference in paper I refereed to dynamic programming solver?

  Speaking generally, we now have two new solvers to play with:
  the single resource solver, and the cluster minimum solver.
  Our mission is to make the best use we can of both.  We can
  run the cluster minimum solver once at the start and have its
  results permanently used throughout the rest of the solve.  And
  we can use KheSingleResourceSolverBest in conjunction with a
  balance solver to select a best solution from the single
  resource solver, and adopt that solution.  But when should we
  run single resource solving, and which resource(s) should we
  select for single resource solving?  For example:

  * Run single-resource solving on a fixed percentage of the
    resources (with highest workload limits) before time sweep,
    and then omit those resources from the time sweep.

  * Find optimal timetables for several resources over a subset
    of the interval, and use that as the basis for a VLSN search.

  Explore possible uses for the now-working cluster minimum
  solver.  Could it be run just before time sweep?  Could
  the changed minimum limits remain in place for the entire
  solve?  Also look at the solutions we are getting now from
  single resource assignment.  If one resource is already
  assigned, does that change the solve for the others?

  Make the cluster minimum solver take account of history.

  OK, what about this?  Use "extended profile grouping" to group all
  tasks into runs of tasks of the same shift type and domain.  Then
  use resource packing (largest workload resources first) to pack
  the runs into the resources.  Finish off with ejection chains.
  This to replace the current first stage.  Precede profile grouping
  by combinatorial grouping, to get weekend tasks grouped together.  
  Keep a matching at each time, so that unavailable times of other
  resources are taken into account, we want the unassigned tasks at
  every time to be assignable to the unpacked resources at that time.
  At least it's different!

  After INRC2-4-030-1-6291 is done, INRC2-4-035-0-1718 would be good to
  work on.  The current results are 21% worse, giving plenty to get into.

  Event timetables still to do.  Just another kind of dimension?
  But it shows meets, not tasks.

  Ideas:

  * Some kind of lookahead during time sweep that ensures resources
    get the weekends they need?  Perhaps deduce that the max limit
    implies a min limit, and go from there?

  * Swapping runs between three or more resources.  I tried this
    but it seems to take more time than it is worth; it's better
    to give the extra time to ejection chains

  * Ejection beams - K ejection chains being lengthened in
    parallel, if the number of unrepaired defects exceeds K
    we abandon the repair, but while it is less we keep going
    Tried this, it has some interest but does not improve things.

  * Hybridization with simulated annealing:  accept some chains
    that produce worse solutions; gradually reduce the temperature.

  Decided to just pick up where I left off, more or less, and go to
  work on INRC2-4-030-1-6291.  I'm currently solving in just 5.6
  seconds, so it makes a good test.

  Fun facts about INRC2-4-030-1-6291
  ----------------------------------

  * 4 weeks

  * 4 shifts per day:  Early (1), Day (2), Late (3), and Night (4) 
    The number of required ones varies more or less randomly; not
    assigning one has soft cost 30.

  * 30 Nurses:

       4 HeadNurse:  HN_0,  ... , HN_3
      13 Nurse:      NU_4,  ... , NU_16
       8 Caretaker:  CT_17, ... , CT_24
       5 Trainee:    TR_25, ... , TR_29

    A HeadNurse can also work as a Nurse, and a Nurse can also work
    as a Caretaker; but a Caretaker can only work as a Caretaker, and
    a Trainee can only work as a Trainee.  Given that there are no
    limit resources constraints and every task has a hard constraint
    preferring either a HeadNurse, a Nurse, a Caretaker, or a Trainee,
    this makes Trainee assignment an independent problem.

  * 3 contracts: Contract-FullTime (12 nurses), Contract-HalfTime
    (10 nurses), Contract-PartTime (8 nurses).  These determine
    workload limits of various kinds (see below).  There seems
    to be no relationship between them and nurse type.

  * There are unavailable times (soft 10) but they are not onerous

  * Unwanted patterns: [L][ED], [N][EDL], [D][E] (hard), so these
    prohibit all backward rotations.

  * Complete weekends (soft 30)

  * Contract constraints:                   Half   Part   Full    Wt
    ----------------------------------------------------------------
    Number of assignments                   5-11   7-15  15-20*   20
    Max busy weekends                          1      2      2    30
    Consecutive same shift days (Early)      2-5    2-5    2-5    15
    Consecutive same shift days (Day)       2-28   2-28   2-28    15
    Consecutive same shift days (Late)       2-5    2-5    2-5    15
    Consecutive same shift days (Night)      3-5    3-5    3-5    15
    Consecutive free days                    2-5    2-4    2-3    30
    Consecutive busy days                    2-4    3-5    3-5    30
    ----------------------------------------------------------------
    *15-20 is notated 15-22 but more than 20 is impossible.

  Currently giving XUTT a rest for a while.  Here is its to do
  list, prefixed by + characters:

  +Can distinct() be used for distinct times?  Why not?  And also
  +using it for "same location" might work.

  +I've finished university course timetabling, except for MaxBreaks
  +and MaxBlock, which I intend to leave for a while and ponder over
  +(see below).  I've also finished sports scheduling except for SE1
  +"games", which I am waiting on Bulck for but which will not be a
  +problem.

  +MaxBreaks and MaxBlock
  +----------------------

    +These are challenging because they do the sorts of things that
    +pattern matching does (e.g. idle times), but the criterion
    +which determines whether two things are adjacent is different:

      +High school timetabling - adjacent time periods
      +Nurse rostering - adjacent days
      +MaxBreaks and MaxBlock - intervals have gap of at most S.

    +It would be good to have a sequence of blocks to iterate over,
    +just like we have some subsequences to iterate over in high
    +school timetabling and nurse rostering.  Then MaxBreaks would
    +utilize the number of elements in the sequence, and MaxBlock
    +would utilize the duration of each block.

    +We also need to allow for future incorporation of travel time 
    +into MaxBreaks and MaxBlock.  Two events would be adjacent if
    +the amount of time left over after travelling from the first
    +to the second was at most S.

    +Assuming a 15-week semester and penalty 2:

    +MaxBreaks(R, S):

	+<Tree val="sum|15d">
	    +<ForEach v="$day" from="Days">
		+<Tree val="sum:0-(R+1)|2">
		    +<ForEachBlock v="$ms" gap="S" travel="travel()">
			+<AtomicMeetSet e="E" t="$day">
			+<Tree val="1">
		    +</ForEachBlock>
		+</Tree>
	    +</ForEach>
	+</Tree>

    +MaxBlock(M, S):

	+<Tree val="sum|15d">
	    +<ForEach v="$day" from="Days">
		+<Tree val="sum:0-M|2">
		    +<ForEachBlock v="$ms" gap="S" singles="no" travel="travel">
			+<AtomicMeetSet e="E" t="$day">
			+<Tree val="$ms.span:0-M|1s">
		    +</ForEachBlock>
		+</Tree>
	    +</ForEach>
        +</Tree>

    +Actually it might be better if each iteration produced a meet set.
    +We could then ask for span and so forth as usual.  There is also
    +a connection with spacing(a, b).  In fact it would be good to
    +give a general expression which determines whether two
    +chronologically adjacent meets are in the same block.
    +Then we could use "false" to get every meet into a separate
    +block, and then spacing(a, b) would apply to each pair of
    +adjacent blocks in the ordering.  If "block" has the same
    +type as "meet set", we're laughing.

    +I'll let this lie fallow for a while and come back to it.

  +Rather than sorting meets and defining cost functions which
  +are sums, can we iterate over the sorted meets?

  +The ref and expr attributes of time sequences and event sequences
  +do the same thing.

  +There is an example of times with attributes in the section on
  +weighted domain constraints.  Do we want them?  How do they fit
  +with time pattern trees?  Are there weights for compound times?

  +Moved history from Tree to ForEachTimeGroup.  This will be
  +consistent with pattern matching, and more principled, since
  +history in effect extends the range of the iterator.  But
  +what to do about general patterns?  We need to know how each
  +element of the pattern matches through history.

  +Could use tags to identify specific task sets within patterns.

  Install the new version of HSEval on web site, but not until after
  the final PATAT 2020 deadline.

  In the CQ14-13 table, I need to see available workload in minutes.

  Fun facts about instance CQ14-13
  --------------------------------

  * A four-week instance (1Mon to 4Sun) with 18 times per day:

      a1 (1),  a2 (2),  a3 (3),  a4 (4),  a5 (5),
      d1 (6),  d2 (7),  d3 (8),  d4 (9),  d5 (10),
      p1 (11), p2 (12), p3 (13), p4 (14), p5 (15),
      n1 (16), n2 (17), n3 (18)

    There are workloads, presumably in minutes, that vary quite a bit:

      a1 (480),  a2 (480),  a3 (480),  a4 (600),  a5 (720),
      d1 (480),  d2 (480),  d3 (480),  d4 (600),  d5 (720),
      p1 (480),  p2 (480),  p3 (480),  p4 (600),  p5 (720),
      n1 (480),                        n2 (600),  n3 (720)

    480 minutes is an 8-hour shift, 720 minutes is 12 hours.

  * 120 resources, with many hard preferences for certain shifts:

      Preferred-a1 Preferred-a2 Preferred-a3 Preferred-a4 Preferred-a5
      Preferred-d1 Preferred-d2 Preferred-d3 Preferred-d4 Preferred-d5
      Preferred-p1 Preferred-p2 Preferred-p3 Preferred-p4 Preferred-p5
      Preferred-n1 Preferred-n2 Preferred-n3

    although most resources have plenty of choices from this list.
    Anyway this leads to a huge number of prefer resources constraints.

  * There are also many avoid unavailable times constraints, some for
    whole days, many others for individual times; hard and soft.

  * Unwanted patterns (hard).  In these patterns, a stands for
    [a1a2a3a4a5] and so on.

      [d4][a]
      [p5][adp4-5]
      [n1][adp]
      [n2-3][adpn3]
      [d1-3][a1-4]
      [a5d5p1-4][ad]

    This is basically "day off after a sequence of night shifts",
    with some other stuff that probably matters less; a lot of it
    is about the 480 and 720 minute shifts.

  * MaxWeekends (hard) for most resources is 2, for some it is 1 or 3.

  * MaxSameShiftDays (hard) varies a lot, with fewer of the long
    workload shifts allowed.  NB this is not consecutive, this is
    total.  About at most 10 of the shorter, 3 of the longer.
    Doesn't seem very constraining, given that typical workloads
    are 15 or 16 shifts.

  * Many day or shift on requests, soft with varying weights (1-3).

  * Minimum and maximum workload limits in minutes (hard), e.g.

      Minutes           480-minute shifts
      -----------------------------------------------------------
      3120 - 3840
      4440 - 5160
      7440 - 8160        15.5 - 17.0
      7920 - 8640        16.5 - 18.0

    The last two ranges cover the great majority of resources.
    These ranges are quite tight, especially for hard constraints.

  * MinConsecutiveFreeDays 2 (hard) for most resources, 3 (hard)
    for a few.

  * MaxConsecutiveBusyDays 5 (hard) for most resources, 6 (hard)
    for a few.

  * MinConsecutiveBusyDays 2 (hard), for all or most resources.

  Decided to work on CQ14-13 for a while, then tidy up, rerun,
  and submit.

  What does profile grouping do when the minimum limits are
  somewhat different for different resources, and thus spread
  over several constraints?

  INRC1-ML02 would be a good test.  It runs fast and the gap is
  pretty wide at the moment.  Actually I worked on it before (from
  8 November 2019).  It inspired KhePropagateUnavailableTimes.

  Fun facts about INRC1-ML02
  --------------------------

    * 4 weeks 1Fri to 4Thu

    * 4 shifts per day: E (1), L (2), D (3), and N (4).  But there are
      only two D shifts each day, so this is basically a three-shift
      system of Early, Late, and Night shifts.

    * 30 Nurses:
  
        Contract-0  Nurse0  - Nurse7
        Contract-1  Nurse8  - Nurse26
        Contract-2  Nurse27 - Nurse29

    * Many day and shift off requests, all soft 1 but challenging.
      I bet this is where the cost is incurred.

    * Complete weekends (soft 2), no night shift before free
      weekend (soft 1), identical shift types during weekend (soft 1),
      unwanted patterns [L][E], [L][D], [D][N], [N][E], [N][D],
      [D][E][D], all soft 1

    * Contract constraints         Contract-0    Contract-1   Contract-2
      ----------------------------------------------------------------
      Assignments                    10-18        6-14          4-8
      Consecutive busy weekends       2-3     unconstrained     2-3
      Consecutive free days           2-4         3-5           4-6
      Consecutive busy days           3-5         2-4           3-4
      ----------------------------------------------------------------

      Workloads are tight, there are only 6 shifts to spare, or 8 if
      you ignore the overloads in Nurse28 and Nurse29, which both
      GOAL and KHE18x8 have, so presumably they are inevitable.


  Do something about constraints with step cost functions, if only
  so that I can say in the paper that it's done.

  In INRC2-4-030-1-6291, the difference between my 1880 result and
  the LOR17 1695 result is about 200.  About 100 of that is in
  minimum consecutive same shift days defects.  Max working weekends
  defects are another problem, my solution has 3 more of those
  than the LOR17 solution has; at 30 points each that's 90 points.
  If we can improve our results on these defects we will go a long
  way towards closing the gap.

  Grinding down INRC2-4-030-1-6291 from where it is now.  It would
  be good to get a better initial solution from time sweep than I am
  getting now.  Also, there are no same shift days defects in the
  LOR17 solution, whereas there are 

  Perhaps profile grouping could do something unconventional if it
  finds a narrow peak in the profile that really needs to be grouped.

  What about an ejection chain repair, taking the current runs
  as indivisible?

  My chances of being able to do better on INRC2-4-030-1-6291
  seem to be pretty slim.  But I really should pause and make
  a serious attack on it.  After that there is only CQ to go,
  and I have until 30 January.  There's time now and if I don't
  do it now I never will.

  Better to not generate contract (and skill?) resource groups if
  not used.

  Change KHE's general policy so that operations that change
  nothing succeed.  Having them fail composes badly.  The user
  will need to avoid cases that change nothing.

  Are there other modules that could use the task finder?
  Combinatorial grouping for example?  There are no functions
  in khe_task.c that look like task finding, but there are some
  in khe_resource_timetable_monitor.c:

    KheResourceTimetableMonitorTimeAvailable
    KheResourceTimetableMonitorTimeGroupAvailable
    KheResourceTimetableMonitorTaskAvailableInFrame
    KheResourceTimetableMonitorAddProperRootTasks

  KheTaskSetMoveMultiRepair phase variable may be slow, try
  removing it and just doing everything all together.

  Fun facts about COI-Musa
  ------------------------

  * 2 weeks, one shift per day, 11 nurses (skills RN, LPN, NA)

  * RN nurses:  Nurse1, Nurse2, Nurse3,
    LPN nurses: Nurse4, Nurse5, 
    NA nurses:  Nurse6, Nurse7, Nurse8, Nurse9, Nurse10, Nurse11

  Grinding down COI-HED01.  See above, 10 October, for what I've
  done so far.

  It should actually be possible to group four M's together in
  Week 1, and so on, although combinatorial grouping only tries
  up to 3 days so it probably does not realize this.

  Fun facts about COI-HED01
  -------------------------

    * 31 days, 5 shifts per day: 1=M, 2=D, 3=H, 4=A, 5=N

    * Weekend days are different, they use the H shift.  There
      is also something peculiar about 3Tue, it also uses the
      H shift.  It seems to be being treated like a weekend day.
      This point is reflected in other constraints, which treat
      Week 3 as though it had only four days.

    * All demand expressed by limit resources constraints,
      except for the D shift, which has two tasks subject
      to assign resource and prefer resources constraints.
      The other shifts vary between about 7 and 9 tasks.  But
      my new converter avoids all limit resources constraints.

    * There are 16 "OP" nurses and 4 "Temp" nurses.
      Three nurses have extensive sequences of days off.
      There is one skill, "Skill-0", but it contains the
      same nurses as the OP nurses.

    * The constraints are somewhat peculiar, and need attention
      (e.g. how do they affect combinatorial grouping?)
    
        [D][0][not N]  (Constraint:1)
          After a D, we want a day off and then a night shift (OP only).
	  Only one nurse has a D at any one time, so making this should
	  not be very troublesome.

	[not M][D]  (Constraint:2)
	  Prefer M before D (OP only), always seems to get ignored,
	  even in the best solutions.  This is because during the
	  week that D occurs, we can't have a week full of M's.
	  So really this constraint contradicts the others.

	[DHN][MDHAN]  (Constraint:3)
	  Prefer day off after D, H, or N.  Always seems to be
	  satisfied.  Since H occurs only on weekends, plus 3Tue,
	  each resource can work at most one day of the weekend,
	  and if that day is Sunday, the resource cannot work
	  M or A shifts the following week (since that would
	  require working every day).  Sure enough, in the
	  best solution, when an OP nurse works an H shift on
	  a Sunday, the following week contains N shifts and
	  usually a D shift.  And all of the H shifts assigned
	  to Temp nurses are Sunday or 3Tue ones.

	Constraint:4 says that Temp nurses should take H and
	D shifts only.  It would be better expressed by a
	prefer resources constraint but KHE seems happy
	enough with it.

	Constraint:5 says that assigning any shift at all to
	a Temp nurse is to be penalized.  Again, a prefer
	resources constraint would have been better, but at
	present both KHE and the best solution assign 15 shifts
	to Temp nurses, so that's fine.

	The wanted pattern is {M}{A}{ND}{M}{A}{ND}..., where
	{X} means that X only should occur during a week.
	This is for OP nurses only.  It is expressed rather
	crudely:  if 1 M in Week X, then 4 M in Week X.
	This part of it does not apply to N, however; it says
	"if any A in Week X, then at least one N in Week X+1".
	So during N weeks the resource usually has less than
	4 N shifts, and this is its big chance to take a D.

	OP nurses should take at least one M, exactly one D,
	at least one H, at most 2 H, at least one A, at least
	one N.  These constraints are not onerous.

    * Assign resource and prefer resources constraints specify:

        - There is one D shift per day

    * Limit resources constraints specify 

        Weekdays excluding 3Tue

        - Each N shift must have exactly 2 Skill-0 nurses.

	- Each M shift and each A shift must have exactly 4
	  Skill-0 nurses

	- There are no H shifts

	Weekend days, including 3Tue

	- Each H shift must have at least 2 Skill-0 nurses

	- Each H shift must have exactly 4 nurses altogether

	- There are no M, A, or N shifts on 3Tue

	- There are no M, A, or N shifts on weekend days

    * The new converter is expressing all demands with assign
      resource and prefer resources constraints, as follows:

      D shifts:

        <R>NA=s1000:1</R>
	<R>A=s1000:1</R>

	So one resource, any skill.

      H shifts (weekends and 3Tue):

        <R>NA=s1000+NW0=s1000:1</R>
	<R>NA=s1000+NW0=s1000:2</R>
	<R>NA=s1000:1</R>
	<R>NA=s1000:2</R>
	<R>A=s1000:1</R>

	So 2 Skill-0 and 2 arbitrary, as above

      M and A shifts (weekdays not 3Tue):

        <R>NA=s1000+NW0=s1000:1</R>
	<R>NA=s1000+NW0=s1000:2</R>
	<R>NA=s1000+NW0=s1000:3</R>
	<R>NA=s1000+NW0=s1000:4</R>
	<R>W0=s1000:1</R>
	<R>W0=s1000:2</R>
	<R>W0=s1000:3</R>
	<R>W0=s1000:4</R>
	<R>W0=s1000:5</R>

	So exactly 4 Skill-0, no limits on Temp nurses

      N shifts (weekday example)

        <R>NA=s1000+NW0=s1000:1</R>
	<R>NA=s1000+NW0=s1000:2</R>
	<R>W0=s1000:1</R>
	<R>W0=s1000:2</R>
	<R>W0=s1000:3</R>
	<R>W0=s1000:4</R>
	<R>W0=s1000:5</R>

      Exactly 2 Skill-0, no limits on Temp nurses.

  It would be good to have a look at COI-HED01.  It has
  deteriorated and it is fast enough to be a good test.
  Curtois' best is 136 and KHE18x8 is currently at 183.
  A quick look suggests that the main problems are the
  rotations from week to week.

  Back to grinding down CQ14-05.  I've fixed the construction
  problem but with no noticeable effect on solution cost.

  KheClusterBusyTimesConstraintResourceOfTypeCount returns the
  number of resources, not the number of distinct resources.
  This may be a problem in some applications of this function.

  Fun facts about CQ14-05
  -----------------------

    * 28 days, 2 shifts per day (E and L), whose demand is:

           1Mon 1Tue 1Wed 1Thu 1Fri 1Sat 1Sun 2Mon 2Tue 2Wed 2Thu
        ---------------------------------------------------------
        E   5    7    5    6    7    6    6    6    6    6    5
        L   4    4    5    4    3    3    4    4    4    6    4
        ---------------------------------------------------------
        Tot 9   11   10   10   10    9   10   10   10   12    9

      Uncovered demands (assign resources defects) make up the
      bulk of the cost (1500 out of 1543).  Most of this (14 out
      of 15) occurs on the weekends.

    * 16 resources named A, B, ... P.  There is a Preferred-L
      resource group containing {C, D, F, G, H, I, J, M, O, P}.
      The resources in its complement, {A, B, E, K, L, N}, are
      not allowed to take late shifts.

    * Max 2 busy weekends (max 3 for for resources K to P)

    * Unwanted pattern [L][E]

    * Max 14 same-shift days (not consecutive).  Not hard to
      ensure given that resource workload limits are 16 - 18.

    * Many day or shift on requests.  These basically don't
      matter because they have low weight and my current best
      solution has about the same number of them as Curtois'

    * Workload limits (all resources) min 7560, max 8640
      All events (both E and L) have workload 480;
      7560 / 480 = 15.7, 8640 / 480 = 18.0, so every resurce
      needs between 16 and 18 shifts.  The Avail column agrees.

    * Min 2 consecutive free days (min 3 for resources K to P)

    * Max 5 consecutive busy days (max 6 for resources K to P)

    * Curtois' best is 1143.  This represents 2 fewer unassigned
      shifts (costing 100 each) and virtually the same other stuff.

  Try to get CQ14-24 to use less memory and produce better results.
  But start with a smaller, faster CQ14 instance:  CQ14-05, say.

  In Ozk*, there are two skill types (RN and Aid), and each
  nurse has exactly one of those skills.  Can this be used to
  convert the limit resources constraints into assign resource
  and prefer resources constraints?

  Grinding down COI-BCDT-Sep in general.  I more or less lost
  interest when I got cost 184 on the artificial instance, but
  this does include half-cycle repairs.  So more thought needed.
  Could we add half-cycle repairs to the second repair phase
  if the first ended quickly?

  KheCombSolverAddProfileGroupRequirement could be merged with
  KheCombSolverAddTimeGroupRequirement if we add an optional
  domain parameter to KheCombSolverAddTimeGroupRequirement.

  Fun facts about COI-BCDT-Sep
  ----------------------------

    * 4 weeks and 2 days, starting on a Wednesday

    * Shifts: 1 V (vacation), 2 M (morning), 3 A (afternoon), 4 N (night).

    * All cover constraints are limit resources constraints.  But they
      are quite strict and hard.  Could they be replaced by assign
      resource constraints?  (Yes, they have been.)

	  Constraint            Shifts               Limit    Cost
	  --------------------------------------------------------
          DemandConstraint:1A   N                    max 4      10
	  DemandConstraint:2A   all A; weekend M     max 4     100
	  DemandConstraint:3A   weekdays M           max 5     100
	  DemandConstraint:4A   all A, N; weekend M  max 5    hard
	  DemandConstraint:5A   weekdays M           max 6    hard
	  DemandConstraint:6A   all A, N; weekend M  min 3    hard
	  DemandConstraint:7A   all N                min 4      10
	  DemandConstraint:8A   all A; weekend M     min 4     100
	  DemandConstraint:9A   weekday M            min 4    hard
	  DemandConstraint:10A  weekday M            min 5     100
	  --------------------------------------------------------

      Weekday M:   min 4 (hard), min 5 (100), max 5 (100), max 6 (hard),
      Weekend M:   min 3 (hard), min 4 (100), max 4 (100), max 5 (hard) 
      All A:       min 3 (hard), min 4 (100), max 4 (100), max 5 (hard)
      All N:       min 3 (hard), min 4 (10),  max 4 (10),  max 5 (hard)

    * There are day and shift off constraints, not onerous

    * Avoid A followed by M

    * Night shifts are to be assigned in blocks of 3, although
      a four block is allowed to avoid fri N and sat free.  There
      are hard constraints requiring at least 2 and at most 4
      night shifts in a row.

    * At least six days between sequences of N shifts; the
      implementation here could be better, possibly.

    * At least two days off after five consecutive shifts

    * At least two days off after night shift

    * Prefer at least two morning shifts before a vacation period and
      at least one night shift afterwards

    * Between 4 and 8 weekend days

    * At least 10 days off

    * 5-7 A (afternoon) shifts, 5-7 N (night) shifts

    * Days shifts (M and A, taken together) in blocks of exactly 3

    * At most 5 working days in a row.

  Work on COI-BCDT-Sep, try to reduce the running time.  There are
  a lot of constraints, which probably explains the poor result.

  Should we limit domain reduction at the start to hard constraints?
  A long test would be good.

  In khe_se_solvers.c, KheAddInitialTasks and KheAddFinalTasks could
  be extended to return an unassign_r1_ts task set which could then be
  passed on to the double repair.  No great urgency, but it does make
  sense to do this.  But first, let's see whether any instances need it.

  Also thought of a possibility of avoiding repairs during time sweep,
  when the cost blows out too much.  Have to think about it and see if
  it is feasible.

  Take a close look at resource matching.  How good are the
  assignments it is currently producing?  Could it do better?

  Now it is basically the big instances, ERRVH, ERMGH, and MER
  that need attention.  Previously I was working on ERRVH, I
  should go back to that.

  Is lookahead actually working in the way I expect it to?
  Or is there something unexpected going on that is preventing
  it from doing what it has the potential to do?

  UniTime requirements not covered yet:

    Need an efficient way to list available rooms and their
    penalties.  Nominally this is done by task constraints but
    something more concise, which indicates that the domain
    is partitioned, would be better.

    Ditto for the time domain of a meet.

    SameStart distribution constraint.  Place all times
    with the same start time in one time group, have one
    time group for each distinct starting time, and use
    a meet constraint with type count and eval="0-1|...".

    SameTime is a problem because there is not a simple
    partition into disjoint sets of times.  Need some
    kind of builtin function between pairs of times, but
    then it's not clear how this fits in a meet set tree.

    DifferentTime is basically no overlap, again we seem
    to need a binary attribute.

    SameDays and SameWeeks are cluster constraints, the limit
    would have to be extracted from the event with the largest
    number of meets, which is a bit dodgy.

    DifferentDays and DifferentWeeks just a max 1 on each day
    or week.

    Overlap and NotOverlap: need a binary for the amount of
    overlap between two times, and then we can constrain it
    to be at least 1 or at most 0.  NB the distributive law

       overlap(a+b, c+d) = overlap(a, c) + overlap(a, d)
         + overlap(b, c) + overlap(b, d)

    but this nice property is not going to hold for all
    binary attributes.

    Precedence: this is the order events constraint, with
    "For classes that have multiple meetings in a week or
    that are on different weeks, the constraint only cares
    about the first meeting of the class."  No design for
    this yet.

    WorkDay(S): "There should not be more than S time slots
    between the start of the first class and the end of the
    last class on any given day."  This is a kind of avoid
    idle times constraint, applied to events rather than to
    resources (which for us is a detail).
      One task or meet set per day, and then a special function
    (span or something) to give the appropriate measure.  But
    how do you define one day?  By a time group.

    MinGap(G): Any two classes that are taught on the same day
    (they are placed on overlapping days and weeks) must be at
    least G slots apart.  Not sure what to make of this.
    I guess it's overlap(a, b, extension) where extension
    applies to both a and b.

    MaxDays(D): "Given classes cannot spread over more than D days
    of the week".  Just a straight cluster constraint.

    MaxDayLoad(S): "Given classes must be spread over the days
      of the week (and weeks) in a way that there is no more
      than a given number of S time slots on every day."  Just
      a straight limit busy times constraint, measuring durations.
      But not the full duration, rather the duration on one day.

      This is one of several indications that we cannot treat
      a non-atomic time as a unit in all cases.

    MaxBreaks(R,S): "MaxBreaks(R,S) This constraint limits the
      number of breaks during a day between a given set of classes
      (not more than R breaks during a day). For each day of week
      and week, there is a break between classes if there is more
      than S empty time slots in between."  A very interesting
      definition of what it means for two times to be consecutive.

    MaxBlock(M,S): "This constraint limits the length of a block
      of consecutive classes during a day (not more than M slots
      in a block). For each day of week and week, two consecutive
      classes are considered to be in the same block if the gap
      between them is not more than S time slots."  Limit active
      intervals, interpreted using durations rather than times.

  A resource r is busy at some time t if that time overlaps with
  any interval in any meet that r is attending.

  Need a way to define time *groups* to take advantage of symmetries.
  e.g. 1-15{MWF}3 = {1-15M3, 1-15W3, 1-15F3}.  All doubles:
  [Mon-Fr][12 & 23 & 45 & 67 & 78] or something.
  {MWF:<time>} or something.  But what is the whole day anyway?
  All intervals, presumably. {1-15:{MTWRF:1-8}

  See 16 April 2019 for things to do with the XUTT paper.

  It's not clear at the moment how time sweep should handle
  rematching.  If left as is, without lookahead, it might
  well undo all the good work done by lookahead.  But to
  add lookahead might be slow.  Start by turning it off:
  rs_time_sweep_rematch_off=true.  The same problem afflicts
  ejection chain repair during time sweep.  Needs thought.
  Can the lookahead stuff be made part of the solution cost?
  "If r is assigned t, add C to solution cost".  Not easily.
  It is like a temporary prefer resources monitor.

  Here's an idea for a repair:  if a sequence is too short, try
  moving it all to another resource where there is room to make
  it longer.  KheResourceUnderloadAugment will in fact do nothing
  at all in these cases, so we really do need to do something,
  even an ejecting move on that day.

  Working over INRC2-4-030-1-6753 generally, trying to improve
  the ejection chain repairs.  No luck so far.

  Resource swapping is really just resource rematching, only not
  as good.  That is, unless there are limit resources constraints.

  The last few ideas have been too small beer.  Must do better!
  Currently trying to improve KHE18's solutions to INRC2-4-035-2-8875.xml:

    1 = Early, 2 = Day, 3 = Late, 4 = Night
    FullTime: max 2 weekends, 15-22 shifts, consec 2-3 free 3-5 busy
    PartTime: max 2 weekends,  7-15 shifts, consec 2-5 free 3-5 busy
    HalfTime: max 1 weekends,  5-11 shifts, consec 3-5 free 3-5 busy
    All: unwanted [4][123], [3][12], complete weekends, single asst per day
    All: consec same shift days: Early 2-5, Day 2-28, Late 2-5, Night 4-5

    FullTime resources and the number of weekends they work in LOR are:
    
      NU_8 2, NU_9 1, CT_17 1, CT_18 0, CT_20 1, CT_25 1, TR_30 2, TR_32 3

    NB full-time can only work 20 shifts because of max 5 busy then
    min 2 free, e.g. 5-2-5-2-5-2-5-2 with 4*5 = 20 busy shifts.  But
    this as it stands is not viable because you work no weekends.  The
    opposite, 2-5-2-5-2-5-2-5 works 4 weekends which is no good either.
    Ideally you would want 5-2-5-4-5-2-5, which works 2 weekends, but
    the 4 free days are a defect.  More breaks is the only way to
    work 2 weekends, but that means a lower workload again.  This is
    why several of LOR's full-timers are working only 18 hours.  The
    conclusion is that trying to redistribute workload overloads is
    not going to help much.

    Resource types

    HeadNurse (HN_*) can also work as Nurse or Caretaker
    Nurse     (NU_*) can also work as Caretaker
    Caretaker (CT_*) works only as Caretaker
    Trainee   (TR_*) works only as Trainee

  "At least two days off after night shift" - if we recode this,
  we might do better on COI-BCDT-Sep.  But it's surprisingly hard.

  Option es_fresh_visits seems to be inconsistent, it causes
  things to become unvisited when there is an assumption that
  they are visited.  Needs looking into.  Currently commented
  out in khe_sr_combined.c.

  For the future:  time limit storing.  khe_sm_timer.c already
  has code for writing time limits, but not yet for reading.

  Work on time modelling paper for PATAT 2020.  The time model
  is an enabler for any projects I might do around ITC 2019,
  for example modelling student sectioning and implementing
  single student timetabling, so it is important for the future
  and needs to be got right.

  Time sets, time groups, resource sets, and resource groups
  ----------------------------------------------------------

    Thinking about whether I can remove construction of time
    neighbourhoods, by instead offering offset parameters on
    the time set operations (subset, etc.) which do the same.

    Need to use resource sets and time sets a lot more in the
    instance, for the constructed resource and time sets which
    in general have no name.  Maybe replace solution time groups
    and solution resource groups altogether.  But it's not
    trivial, because solution time groups are used by meets,
    and solution resource groups are used by tasks, both for
    handling domains (meet and task bounds).  What about

      typedef struct khe_time_set_rec {
          SSET elems;
      } KHE_TIME_SET;

    with SSET optimized by setting length to -1 to finalize.
    Most of the operations would have to be macros which
    add address-of operators in the style of SSET itself.

       KHE_TIME_SET KheTimeSetNeighbour(KHE_TIME_SET ts, int offset);

    would be doable with no memory allocation and one binary
    search (which could be optional for an internal version).

    I'm leaving this lie for now, something has to be done
    here but I'm not sure what, and there is no great hurry.

  There is a problem with preparing once and solving many times:
  adjustments for limit resources monitors depend on assignments
  in the vicinity, which may vary from one call to another.  The
  solution may well be simply to document the issue.

  At present resource matching is grouping then ungrouping during
  preparation, then grouping again when we start solving.  Can this
  be simplified?  There is a mark in the way.

  Document sset (which should really be khe_sset) and khe_set.

  I'm slightly worried that the comparison function for NRC
  worker constraints might have lost its transitivity now that
  history_after is being compared in some cases but not others.

  Look at the remaining special cases in all.map and see if some
  form of condensing can be applied to them.

  Might be a good idea to review the preserve_existing option in
  resource matching.  I don't exactly understand it at the moment.

  There seem to be several silly things in the current code that are
  about statistics.  I should think about collecting statistics in
  general, and implement something.  But not this time around.

  KheTaskFirstUnFixed is quite widely used, but I am beginning to
  are the same as mine (which GOAL's are not)?  If so I need
  to compare my results with theirs.  The paper is in the 2012
  PATAT proceedings, page 254.  Also it gives this site:

    https://www.kuleuven-kulak.be/nrpcompetition/competitor-ranking

  Can I find the results from the competition winner?  According to
  Santos et al. this was Valouxis et al, but their paper is in EJOR.

  Add code for limit resources monitors to khe_se_secondary.c.

  In KheClusterBusyTimesAugment, no use is being made of the
  allow_zero option at the moment.  Need to do this some time.

  Generalize the handling of the require_zero parameter of
  KheOverloadAugment, by allowing an ejection tree repair
  when the ejector depth is 1.  There is something like
  this already in KheClusterOverloadAugment, so look at
  that before doing anything else.

  There is an "Augment functions" section of the ejection chains
  chapter of the KHE guide that will need an update - do it last.

  (KHE) What about a general audit of how monitors report what
  is defective, with a view to finding a general rule for how
  to do this, and unifying all the monitors under that rule?
  The rule could be to store reported_deviation, renaming it
  to deviation, and to calculate a delta on that and have a
  function which applies the delta.  Have to look through all
  the monitors to see how that is likely to pan out.  But the
  general idea of a delta on the deviation does seem to be
  right, given that we want evaluation to be incremental.

  (KHE) For all monitors, should I include attached and unattached
  in the deviation function, so that attachment and unattachment
  are just like any other update functions?

  Ejection chains idea:  include main loop defect ejection trees
  in the major schedule, so that, at the end when main loop defects
  have resisted all previous attempts to repair them, we can try
  ejection trees on each in turn.  Make one change, produce several
  defects, and try to repair them all.  A good last resort?

  Ejection chains idea:  instead of requiring an ejection chain
  to improve the solution by at least (0, 1), require it to
  improve it by a larger amount, at first.  This will run much
  faster and will avoid trying to fix tiny problems until there
  is nothing better to do.  But have I already tried it?  It
  sounds a lot like es_limit_defects.
