The New Sudoku Players' Forum

[Mathimagics]

Do we really know that it's complete, or only that nothing new was found in the 1,365,506,043 grids mentioned here ?

Ah, a very good point! "Nothing new" only refers to the 1.365 billion grids that were explicitly tested.

So indeed, there could well be some "no NTA, no 668" cases hidden amongst the already-resolved grids ...

... and finding them would need a full pass through the catalog.

[Mathimagics]

At the end of week 1, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602
                ------------
               1,119,753,700


That's the "honeymoon" week, of course ...

[Mathimagics]

Ok, the options are "cloud" and "peer-to-peer".

First you should compress your batches into a single zip file, you can then upload this to a cloud server (eg: coloin uses Google Drive) and send me the download link.

Or, you can use a torrent client (eg uTorrent) to create a torrent file for the zip file, and send me the torrent file.

I will PM you with my email address.

[Mathimagics]

That's fine by me ...

[Mathimagics]

Hi blue!

Thanks for the Blue19 binaries, I will road-test it tonight.

I discovered two (bad) things about the BlueMagic app, for LCT-20, when I was converting it.
First, it had a bug:
Only 4 of the 6 bands/stacks were given an opertunity (if needed), to be the "8-clue" band for a 668 puzzle.
Fortunately (and amazingly), I guess ... the "if needed" part of that, never kicked in

Perhaps 20C puzzles are sufficiently numerous that it rarely needs to look beyond the first band?

Second: I accidentally had part of the code using the 256-bit (YMM) registers.
I don't think it sped the code up, and unfortunatly it made the cores run hotter.
The result would be that "Turbo-boost", likely didn't work as well as it should have

I hadn't noticed any core-cooking, but will seek compensation anyway!

[Mathimagics]

I have tested Blue19, all is well there, and I am getting 39-40 grids/sec at the moment, which is more or less as originally expected based on blue's figures.

blue's "bad news" is not unexpected, and, I think, not really that bad. Blue19 will take longer per grid on average as the catalog is "pruned", sure, but average times will depend on the proportion of high-cost grids.

The bias in the {-1,+1} morph method is theoretically clear, but its actual impact on the LCT-19 processing is less so.

What is clear is that at some point, Blue19 will be more efficient than the morph method. Perhaps not with 25% of the grids unresolved, as was the case with LCT-20, but probably later, say around the 10-15% mark.

It is also possible that the morph method will run out of steam sooner than LCT-20 did, and this will probably shift the predicted crossover back towards 25%.

Only time will tell ...

[Mathimagics]

At the end of week 2, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602  Week 1
                 917,301,757  Week 2
               =============
               2,037,055,457


Many thanks for the contributions from coloin and 1to9only!

Most of those contributions have been delivered in the past week, so the closeness of the figures for Week1 and Week 2 is deceptive.

The average yield at the end of week 1 was just over 50%, but now, at the end of week 2, it has declined to around 37%.

[Mathimagics]

For dinev, here is the grid canonicaliser tool.

Usage: canonicalise infile [outfile]

This tool simply takes an input file of solution grids, converts them to CF (using dobrichev's canonical form functions from GridChecker) and writes the ED grids to outfile if specified, otherwise to standard output.

Canonicalise.zip

Canonicalise tool (Win64)

(45.19 KiB) Downloaded 189 times

[Mathimagics]

Done!

[Mathimagics]

At the end of week 3, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602  Week 1
                 917,301,757  Week 2
                 451,939,607  Week 3
               =============
               2,488,995,064


New grid yields for the morph workers is currently about 25%.

[Mathimagics]

At the end of week 4, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602  Week 1
                 917,301,757  Week 2
                 451,939,607  Week 3
                 507,056,483  Week 4
               =============
               2,996,051,547


New grid yields are currently 23%.

[Mathimagics]

At the end of week 5, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602  Week 1
                 917,301,757  Week 2
                 451,939,607  Week 3
                 507,056,483  Week 4
                 358,256,360  Week 5
               =============
               3,354,307,907


New grid yields are now 14-16%. I think that after one more week it will be time to shut down the morph workers and switch to explicit grid testing using blue's Find19C function.

[Mathimagics]

New grid yields are now 14-16%. I think that after one more week it will be time to shut down the morph workers and switch to explicit grid testing using blue's Find19C function.

blue has produced an improved version of Gen19C, which optimises the {-1,+1} morph processing, and thereby increases grid generation by roughly 4 times (!!).

This means that the morph workers have a considerably extended shelf life, and will be worth running for some time to come. For coloin and 1to9only, here is a new 64-bit version of Gen19C.exe.

<zip file removed> (obsolete)

[Mathimagics]

At the end of week 6, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602  Week 1
                 917,301,757  Week 2
                 451,939,607  Week 3
                 507,056,483  Week 4
                 358,256,360  Week 5
                 543,137,278  Week 6
               =============
               3,897,445,185 


Yields are down to 12%, but batch production is up 350% (nice job, blue!), and so this week's new-grid count actually went up ...

Thanks to coloin and 1to9only for grinding out their batches ... keep it up, guys!

[Mathimagics]

Another week, another progress report!

At the end of week 7, we have:

Code: Select all

 known 17-19C:   144,490,098
 added    19C:   975,263,602  Week 1
                 917,301,757  Week 2
                 451,939,607  Week 3
                 507,056,483  Week 4
                 358,256,360  Week 5
                 543,137,278  Week 6
                 467,361,125  Week 7
               =============
               4,364,806,310  Total resolved
               1,107,924,228  Unresolved


The following table lists the last 12 bulk update runs (the past two weeks). Each update corresponds to 128 batch files produced by Gen19C workers at the main data mining sites (myself, coloin, 1to9only). "Grids" is the total ED grids (in millions) in the batches, "New" is the number of new grids resolved, "Yield" is the ratio New/Grids:

Code: Select all

       Date  Grids   New   Yield
  ------------------------------
  1  21 Sep    731   124   0.170
  2  23 Sep    746   121   0.162
  3  24 Sep    770   108   0.140
  4  25 Sep    772    98   0.126
  5  26 Sep    790    92   0.117
  6  27 Sep    791    84   0.106
  7  28 Sep    781    75   0.095
  8  28 Sep    793    67   0.084
  9  29 Sep    791    63   0.080
 10  30 Sep    792    60   0.076
 11   1 Oct    782    54   0.069
 12   2 Oct    789    49   0.062


Another measure of particular relevance is the cost of explicit grid testing (via blue's Find19C function). Just as blue predicted, as the pool of unresolved grids is reduced, the concentration of grids with low 19C puzzle counts increases, and so does the average time per grid for Find19C.

Two weeks ago, with 2.2 billion (40%) grids unresolved, sampling suggested that the Find19C cost per grid was ~25ms (40 grids/sec). Today, with 1.1 billion (20%) grids unresolved, the cost has risen to ~38ms (25 grids/sec).

Is it time, then, given the low yields, to switch to explicit grid testing?

The answer is, I think, no! One Gen19C worker can produce roughly 100 million grids a day. Find19C on today's figures can resolve 2 million a day, so Gen19C at 6% yield is still ~3x more productive.