Low/Hi Clue Thresholds

Everything about Sudoku that doesn't fit in one of the other sections

Re: Low/Hi Clue Thresholds

Postby Mathimagics » Tue Aug 13, 2019 7:44 am

Thanks blue!

I had arrived at the same conclusion myself and have patched all the files. Sloppy work on my part ... :oops:

The patched source is available above, could you cast your eye over it? I have verified it with a small batch run (see line #17 in the source) and it works ok.

Cheers
MM

PS: LCT-20 BlueMagic processing will be complete in 24 hours ...
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

Re: Low/Hi Clue Thresholds

Postby blue » Tue Aug 13, 2019 8:29 am

Mathimagics wrote:The patched source is available above, could you cast your eye over it?

It's better, but:
Code: Select all
      nx++; if (nx >= MAXN) {nx = MAXN; tFull = 1; return;}  // stop adding
      memcpy(XT, NT, 166); XT += 166;

should be more like:
Code: Select all
      memcpy(XT, NT, 166); XT += 166;
      if (++nx >= TBLSIZE) {tFull = 1; return;}  // stop adding

or (safer, I guess):
Code: Select all
      nx++; if (nx > TBLSIZE) {nx = TBLSIZE; tFull = 1; return;}  // stop adding
      memcpy(XT, NT, 166); XT += 166;

... TBLSIZE, not MAXN ... and either > instead of >=, or increment & check *after* adding the puzzle.

There are the other issues too:
1) int endM = NMT ? 0 : 1; // on entry
2) after the "F(nr, NNT)" loop, doing something like
Code: Select all
while (!tFull && mr < NMT) { AddPuzzleM(); mr++; MT += 166; }

---

Mathimagics wrote:PS: LCT-20 BlueMagic processing will be complete in 24 hours ...

Sounds good !
Still no new grids without a 668 puzzle ?

I've got something due to complete in ~22 hours, myself.

Then I'll work on an LCT-19 replacement for BlueMagic.
[ The tests I did before, ran in passes: "test grids; set #define's for next pass; recompile; loop". ]
blue
 
Posts: 869
Joined: 11 March 2013

Re: Low/Hi Clue Thresholds

Postby Mathimagics » Tue Aug 13, 2019 11:15 am

blue wrote:Still no new grids without a 668 puzzle ?

No, not a sausage! Looks very much like you already have the complete list ... (and that there are only those 4 cases of "No20C" grids)
blue wrote:Then I'll work on an LCT-19 replacement for BlueMagic.

Excellent!
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

Re: Low/Hi Clue Thresholds

Postby 1to9only » Tue Aug 13, 2019 1:10 pm

I have 4 new Workers running. The next few hours should produce Batches!!

However, I've been able to reproduce the crash on BOTH yesterday's old code and today's new code, using the following values:
Code: Select all
#define MAXN     26056
#define TABSIZE 0x6000

The crash occurs within a few minutes of starting. [This is on a different machine from yesterday's.]

MAXN=12936648 and MAXN=26056 both allocate memory that leaves 80 unused bytes at the end of the allocated block. ConsolidateList() compares 81 bytes (less bytes on mismatches so may not get out of bounds) and then copies 166 bytes. This could lead to a crash if the MAXN+1 item is processed. I'm not even sure if this is a likely scenario!

My way forward will be to dump the allocated tables to files before calling ConsolidateList().

If a crash occurs, I'll have a modified program to load the tables from files before calling ConsolidateList(), and debugging this.
.
1to9only
 
Posts: 1093
Joined: 04 April 2018

Re: Low/Hi Clue Thresholds

Postby Mathimagics » Tue Aug 13, 2019 2:15 pm

1to9only: I think that I have changed the source since you last downloaded it.

There should be no spare bytes, as all table allocations are done in multiples (via the defines MAXN and TABSIZE) of 166 bytes. This is the standard table entry format:

  • ED grid (81)
  • a colon + a space (2)
  • 19C puzzle (81)
  • CR + LF (2)

Sort/merge comparisons are done only on the grid portion. blue pointed out that I was using the wrong comparison (my shame :oops: ) in the AddPuzzleM/N procs, which I have now corrected and posted above. These were the root cause of the crashes, I think.

Cheers,
MM

[EDIT] Not all batch-runs fill the larger table, but here is the log of one which I did overnight that does fill it (in pass #10), and lives to tell the tale:
Gen19C log: Show
Code: Select all
21:57:31: Generating initial seeds
  1: .23...7......89........1.........46...8...2..9.1.........64..2....32.6..8........
  2: 12....7.......9.3....2......8...4.9..74..............5....6.4..5..72......9...3..
  3: .2........5.7....3...1..5.....36..........27..8....4......9...8....2.9...31.4....
  4: ......7.9456.8......8......2.....4.....3...1....9...3......5....71.....6....34...
  5: ....5..........1...8.1....4.6...4..3...92..5..3.....2....6...785...1......2......
  6: ....56.....718.....9......4.......6......2.97..1...8......1....5..9.....7.6.....2
  7: .23...7...5............1.46.7.5.........4..6...8...3..3....4...6......9....3.7...
  8: 12............9..66...3.4.5..5.....4..1972......8.......4.........6...2..9.......
  9: .2....7..4....92..6......1......7...5.16...........9.........6..7..42......3.1.5.
 10: 1.3.5......7.8...6........4...6......7....8.39....5....6.9..5......3.9....4......
22:02:19: Added 10 new seeds
22:02:19: Processing new items - pass #1
22:02:19: Items in = 10, items out = 42
22:02:19: Sort/merge new items
22:02:19: Pass complete: ED grids/puzzles =        43
22:02:19: Processing new items - pass #2
22:02:19: Items in = 42, items out = 162
22:02:19: Sort/merge new items
22:02:19: Pass complete: ED grids/puzzles =       162
22:02:19: Processing new items - pass #3
22:02:22: Items in = 162, items out = 559
22:02:22: Sort/merge new items
22:02:22: Pass complete: ED grids/puzzles =       559
22:02:22: Processing new items - pass #4
22:02:29: Items in = 559, items out = 2282
22:02:29: Sort/merge new items
22:02:29: Pass complete: ED grids/puzzles =      2012
22:02:29: Processing new items - pass #5
22:02:53: Items in = 2012, items out = 11025
22:02:53: Sort/merge new items
22:02:53: Pass complete: ED grids/puzzles =      7815
22:02:53: Processing new items - pass #6
22:04:26: Items in = 7815, items out = 51085
22:04:26: Sort/merge new items
22:04:26: Pass complete: ED grids/puzzles =     32955
22:04:26: Processing new items - pass #7
22:11:03: Items in = 32955, items out = 242899
22:11:03: Sort/merge new items
22:11:03: Pass complete: ED grids/puzzles =    150805
22:11:03: Processing new items - pass #8
22:41:23: Items in = 150805, items out = 1189410
22:41:24: Sort/merge new items
22:41:24: Pass complete: ED grids/puzzles =    723467
22:41:24: Processing new items - pass #9
01:04:53: Items in = 723467, items out = 5978588
01:04:59: Sort/merge new items
01:05:01: Pass complete: ED grids/puzzles =   3572019
01:05:01: Processing new items - pass #10
06:08:12: Items in = 1544485, items out = 12936647
06:09:36: Sort/merge new items
06:09:42: Pass complete: ED grids/puzzles =   8388607
06:09:42: Creating batch file
06:09:45:  Waiting for Batch# lock
        1 file(s) moved.
06:09:45:  Saved as Batch000011.txt
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

LCT-20 Completion

Postby Mathimagics » Wed Aug 14, 2019 12:27 pm

LCT-20 Final Report

Code: Select all
     17C:      46300
     18C:   10658721
     19C:  133785077
     20C: 5328240436
     21C:          4
          ----------
 Puzzles: 5472730538 ( 100.00%)


So, every ED grid (except for 4) has a 20C puzzle, the only exceptions being those 4 grids previously identified by blue and confirmed by dobrichev:

Code: Select all
123456789456789123789123456214897365365214897897365214541632978632978541978541632
123456789456789123789132465218967534564213978937548216391875642645321897872694351
123456789457189326689327154216534897745891632938672541361245978574918263892763415
123456789457189326689327154216534897745891632938672541392765418574918263861243975


blue's discovery of these grids was originally reported here: Grids containing a 21 but no 20

Also, blue's table of "668 failures" is complete, no new cases were discovered.

A pass of the catalog was made to check for redundant clues in the recorded puzzle, and this resulted in about 1/40 of 20C's being "promoted" to 19C, and also turned up a few million extra 18C's.

A big thank you to blue for the BlueMagic20 tool!

And so, onward with LCT-19 ...
Last edited by Mathimagics on Wed Aug 14, 2019 6:29 pm, edited 3 times in total.
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

Re: Low/Hi Clue Thresholds

Postby dobrichev » Wed Aug 14, 2019 3:08 pm

Well done!!!
dobrichev
2016 Supporter
 
Posts: 1733
Joined: 24 May 2010

Re: LCT-20 Completion

Postby blue » Wed Aug 14, 2019 6:11 pm

Hi Mathimagics,

Great work :!:

Mathimagics wrote:Also, blue's table of "668 failures" is complete, no new cases were discovered.

Do we really know that it's complete, or only that nothing new was found in the 1,365,506,043 grids mentioned here ?

It's just a question.
If we don't know that it's complete, I'm not suggesting that you spend any time on the issue.

Cheers,
Blue.
blue
 
Posts: 869
Joined: 11 March 2013

Re: LCT-20 Completion

Postby Mathimagics » Wed Aug 14, 2019 6:50 pm

blue wrote:Do we really know that it's complete, or only that nothing new was found in the 1,365,506,043 grids mentioned here ?


Ah, a very good point! "Nothing new" only refers to the 1.365 billion grids that were explicitly tested.

So indeed, there could well be some "no NTA, no 668" cases hidden amongst the already-resolved grids ...

... and finding them would need a full pass through the catalog. :cry:
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

LCT-19 in progress

Postby coloin » Thu Aug 15, 2019 7:57 pm

Yep more stability in the program and a needed RAM upgrade means
That I have workers in the background churning out 19C

x3 on gen19c
x2 on gen19j

returning home from a trip - have got 19 batch files to zip and transfer ....

C
coloin
 
Posts: 1813
Joined: 05 May 2005

LCT-19 Progress

Postby Mathimagics » Wed Aug 21, 2019 8:15 am

At the end of week 1, we have:
Code: Select all
 known 17-19C:   144,490,098
 added    19C:   975,263,602
                ------------
               1,119,753,700


That's the "honeymoon" week, of course ... ;)
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

Re: Low/Hi Clue Thresholds

Postby 1to9only » Wed Aug 21, 2019 8:49 am

Mathimagics wrote:I will discuss batch delivery options in a separate post.

Is this sorted? I have about 6.3GB of uncompressed batch data from a week ago. PC presently on patterns game duty. I don't want to keep filling up the disk space if I've no way of offloading some batches.
1to9only
 
Posts: 1093
Joined: 04 April 2018

Re: Low/Hi Clue Thresholds

Postby Mathimagics » Wed Aug 21, 2019 9:06 am

Ok, the options are "cloud" and "peer-to-peer".

First you should compress your batches into a single zip file, you can then upload this to a cloud server (eg: coloin uses Google Drive) and send me the download link.

Or, you can use a torrent client (eg uTorrent) to create a torrent file for the zip file, and send me the torrent file.

I will PM you with my email address.
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

Re: Low/Hi Clue Thresholds

Postby 1to9only » Fri Aug 23, 2019 6:22 pm

The last time I had the Workers toiling away, I ended up hitting Ctrl-C to stop them and reclaim the Computer for other uses. The Batch data was lost. I'm going to modify my Workers to check for the STOPB signal after each pass and exit gracefully. No Batch data will be lost, but the Batches will be smaller.
Code: Select all
   while (1) {
      Pass++;
      CheckPause();
      LogEventN("Processing new items - pass #", Pass);
      ...
      gf = fopen("STOPB", "r");
      if (gf) { fclose(gf); break; }
   }
1to9only
 
Posts: 1093
Joined: 04 April 2018

Re: Low/Hi Clue Thresholds

Postby Mathimagics » Sat Aug 24, 2019 5:12 am

That's fine by me ...
User avatar
Mathimagics
2017 Supporter
 
Posts: 1293
Joined: 27 May 2015
Location: Canberra

PreviousNext

Return to General