The New Sudoku Players' Forum

[Mathimagics]

Thanks blue!

I had arrived at the same conclusion myself and have patched all the files. Sloppy work on my part ...

The patched source is available above, could you cast your eye over it? I have verified it with a small batch run (see line #17 in the source) and it works ok.

Cheers
MM

PS: LCT-20 BlueMagic processing will be complete in 24 hours ...

[blue]

The patched source is available above, could you cast your eye over it?

It's better, but:

Code: Select all

      nx++; if (nx >= MAXN) {nx = MAXN; tFull = 1; return;}  // stop adding
      memcpy(XT, NT, 166); XT += 166;

should be more like:

Code: Select all

      memcpy(XT, NT, 166); XT += 166;
      if (++nx >= TBLSIZE) {tFull = 1; return;}  // stop adding

or (safer, I guess):

Code: Select all

      nx++; if (nx > TBLSIZE) {nx = TBLSIZE; tFull = 1; return;}  // stop adding
      memcpy(XT, NT, 166); XT += 166;

... TBLSIZE, not MAXN ... and either > instead of >=, or increment & check *after* adding the puzzle.

There are the other issues too:
1) int endM = NMT ? 0 : 1; // on entry
2) after the "F(nr, NNT)" loop, doing something like

Code: Select all

while (!tFull && mr < NMT) { AddPuzzleM(); mr++; MT += 166; }

---

PS: LCT-20 BlueMagic processing will be complete in 24 hours ...

Sounds good !
Still no new grids without a 668 puzzle ?

I've got something due to complete in ~22 hours, myself.

Then I'll work on an LCT-19 replacement for BlueMagic.
[ The tests I did before, ran in passes: "test grids; set #define's for next pass; recompile; loop". ]

[Mathimagics]

Still no new grids without a 668 puzzle ?

No, not a sausage! Looks very much like you already have the complete list ... (and that there are only those 4 cases of "No20C" grids)

Then I'll work on an LCT-19 replacement for BlueMagic.

Excellent!

[1to9only]

I have 4 new Workers running. The next few hours should produce Batches!!

However, I've been able to reproduce the crash on BOTH yesterday's old code and today's new code, using the following values:

Code: Select all

#define MAXN     26056
#define TABSIZE 0x6000


The crash occurs within a few minutes of starting. [This is on a different machine from yesterday's.]

MAXN=12936648 and MAXN=26056 both allocate memory that leaves 80 unused bytes at the end of the allocated block. ConsolidateList() compares 81 bytes (less bytes on mismatches so may not get out of bounds) and then copies 166 bytes. This could lead to a crash if the MAXN+1 item is processed. I'm not even sure if this is a likely scenario!

My way forward will be to dump the allocated tables to files before calling ConsolidateList().

If a crash occurs, I'll have a modified program to load the tables from files before calling ConsolidateList(), and debugging this.
.

[Mathimagics]

1to9only: I think that I have changed the source since you last downloaded it.

There should be no spare bytes, as all table allocations are done in multiples (via the defines MAXN and TABSIZE) of 166 bytes. This is the standard table entry format:

• ED grid (81)
• a colon + a space (2)
• 19C puzzle (81)
• CR + LF (2)

Sort/merge comparisons are done only on the grid portion. blue pointed out that I was using the wrong comparison (my shame ) in the AddPuzzleM/N procs, which I have now corrected and posted above. These were the root cause of the crashes, I think.

Cheers,
MM

[EDIT] Not all batch-runs fill the larger table, but here is the log of one which I did overnight that does fill it (in pass #10), and lives to tell the tale:

Gen19C log: Show

Code: Select all

21:57:31: Generating initial seeds
  1: .23...7......89........1.........46...8...2..9.1.........64..2....32.6..8........
  2: 12....7.......9.3....2......8...4.9..74..............5....6.4..5..72......9...3..
  3: .2........5.7....3...1..5.....36..........27..8....4......9...8....2.9...31.4....
  4: ......7.9456.8......8......2.....4.....3...1....9...3......5....71.....6....34...
  5: ....5..........1...8.1....4.6...4..3...92..5..3.....2....6...785...1......2......
  6: ....56.....718.....9......4.......6......2.97..1...8......1....5..9.....7.6.....2
  7: .23...7...5............1.46.7.5.........4..6...8...3..3....4...6......9....3.7...
  8: 12............9..66...3.4.5..5.....4..1972......8.......4.........6...2..9.......
  9: .2....7..4....92..6......1......7...5.16...........9.........6..7..42......3.1.5.
 10: 1.3.5......7.8...6........4...6......7....8.39....5....6.9..5......3.9....4......
22:02:19: Added 10 new seeds
22:02:19: Processing new items - pass #1
22:02:19: Items in = 10, items out = 42
22:02:19: Sort/merge new items
22:02:19: Pass complete: ED grids/puzzles =        43
22:02:19: Processing new items - pass #2
22:02:19: Items in = 42, items out = 162
22:02:19: Sort/merge new items
22:02:19: Pass complete: ED grids/puzzles =       162
22:02:19: Processing new items - pass #3
22:02:22: Items in = 162, items out = 559
22:02:22: Sort/merge new items
22:02:22: Pass complete: ED grids/puzzles =       559
22:02:22: Processing new items - pass #4
22:02:29: Items in = 559, items out = 2282
22:02:29: Sort/merge new items
22:02:29: Pass complete: ED grids/puzzles =      2012
22:02:29: Processing new items - pass #5
22:02:53: Items in = 2012, items out = 11025
22:02:53: Sort/merge new items
22:02:53: Pass complete: ED grids/puzzles =      7815
22:02:53: Processing new items - pass #6
22:04:26: Items in = 7815, items out = 51085
22:04:26: Sort/merge new items
22:04:26: Pass complete: ED grids/puzzles =     32955
22:04:26: Processing new items - pass #7
22:11:03: Items in = 32955, items out = 242899
22:11:03: Sort/merge new items
22:11:03: Pass complete: ED grids/puzzles =    150805
22:11:03: Processing new items - pass #8
22:41:23: Items in = 150805, items out = 1189410
22:41:24: Sort/merge new items
22:41:24: Pass complete: ED grids/puzzles =    723467
22:41:24: Processing new items - pass #9
01:04:53: Items in = 723467, items out = 5978588
01:04:59: Sort/merge new items
01:05:01: Pass complete: ED grids/puzzles =   3572019
01:05:01: Processing new items - pass #10
06:08:12: Items in = 1544485, items out = 12936647
06:09:36: Sort/merge new items
06:09:42: Pass complete: ED grids/puzzles =   8388607
06:09:42: Creating batch file
06:09:45:  Waiting for Batch# lock
        1 file(s) moved.
06:09:45:  Saved as Batch000011.txt


[Mathimagics]

LCT-20 Final Report

Code: Select all

     17C:      46300
     18C:   10658721
     19C:  133785077
     20C: 5328240436
     21C:          4
          ----------
 Puzzles: 5472730538 ( 100.00%)


So, every ED grid (except for 4) has a 20C puzzle, the only exceptions being those 4 grids previously identified by blue and confirmed by dobrichev:

Code: Select all

123456789456789123789123456214897365365214897897365214541632978632978541978541632
123456789456789123789132465218967534564213978937548216391875642645321897872694351
123456789457189326689327154216534897745891632938672541361245978574918263892763415
123456789457189326689327154216534897745891632938672541392765418574918263861243975


blue's discovery of these grids was originally reported here: Grids containing a 21 but no 20

Also, blue's table of "668 failures" is complete, no new cases were discovered.

A pass of the catalog was made to check for redundant clues in the recorded puzzle, and this resulted in about 1/40 of 20C's being "promoted" to 19C, and also turned up a few million extra 18C's.

A big thank you to blue for the BlueMagic20 tool!

And so, onward with LCT-19 ...

[dobrichev]

Well done!!!

[blue]

Hi Mathimagics,

Great work

Also, blue's table of "668 failures" is complete, no new cases were discovered.

Do we really know that it's complete, or only that nothing new was found in the 1,365,506,043 grids mentioned here ?

It's just a question.
If we don't know that it's complete, I'm not suggesting that you spend any time on the issue.

Cheers,
Blue.

[Mathimagics]

Do we really know that it's complete, or only that nothing new was found in the 1,365,506,043 grids mentioned here ?

Ah, a very good point! "Nothing new" only refers to the 1.365 billion grids that were explicitly tested.

So indeed, there could well be some "no NTA, no 668" cases hidden amongst the already-resolved grids ...

... and finding them would need a full pass through the catalog.

[coloin]

Yep more stability in the program and a needed RAM upgrade means
That I have workers in the background churning out 19C

x3 on gen19c
x2 on gen19j

returning home from a trip - have got 19 batch files to zip and transfer ....

C

[Mathimagics]

At the end of week 1, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602
                ------------
               1,119,753,700


That's the "honeymoon" week, of course ...

[1to9only]

I will discuss batch delivery options in a separate post.

Is this sorted? I have about 6.3GB of uncompressed batch data from a week ago. PC presently on patterns game duty. I don't want to keep filling up the disk space if I've no way of offloading some batches.

[Mathimagics]

Ok, the options are "cloud" and "peer-to-peer".

First you should compress your batches into a single zip file, you can then upload this to a cloud server (eg: coloin uses Google Drive) and send me the download link.

Or, you can use a torrent client (eg uTorrent) to create a torrent file for the zip file, and send me the torrent file.

I will PM you with my email address.

[1to9only]

The last time I had the Workers toiling away, I ended up hitting Ctrl-C to stop them and reclaim the Computer for other uses. The Batch data was lost. I'm going to modify my Workers to check for the STOPB signal after each pass and exit gracefully. No Batch data will be lost, but the Batches will be smaller.

Code: Select all

   while (1) {
      Pass++;
      CheckPause();
      LogEventN("Processing new items - pass #", Pass);
      ...
      gf = fopen("STOPB", "r");
      if (gf) { fclose(gf); break; }
   }


[Mathimagics]

That's fine by me ...