The New Sudoku Players' Forum

[Paquita]

I am thinking of a way to split such a large file, for the future maintenance if I am going to do that. Maybe an idea to split by number of clues, that is an invariant given for the minimals.
I have the Sqlite database but that stops at 10.000.000 records

[champagne]

Or maybe I already have the rates if these are from your large files scanprb.

I have most of them, but not all due to the ED canonical morphing.
I think better for the safety to run again the rating.
For ,the next updates, my answer is in the following point

I am thinking of a way to split such a large file, for the future maintenance if I am going to do that. Maybe an idea to split by number of clues, that is an invariant given for the minimals.
I have the Sqlite database but that stops at 10.000.000 records

My choice has been to write the missing code to use only gsort and text files, The sequence for an update is something as

find minimal canonical of the file to process
split the file TH threat and others

For each subfile
find the new minimal canonical (start is puzzle plus name)
add the number of clues
add the rating (from scratch for these new puzzles)
add the TH assumed threat, again for the new
expand the new and merge with the file of old expanded
clean expanded with subsets
extract new expanded and old expanded still valid
rate and add the THassumed threat on the new expanded

In this sequence, we need smal split/merge operations; easy things with a tailor made code.

But I agree that analysis on the final file is easier using a database tool and then, a split on any criteria can help

[Paquita]

Thank you champagne

You put a lot of work in it! Not just the minimals but also the connected expands and their minimals
What is "clean expanded with subsets"?
And the name of the puzzle, is that the string made of the solution?

[champagne]

Thank you champagne

You put a lot of work in it! Not just the minimals but also the connected expands and their minimals
What is "clean expanded with subsets"?
And the name of the puzzle, is that the string made of the solution?

You have more details in these 2 threads

http://forum.enjoysudoku.com/the-hardest-sudoku-3-player-view-t46079.html
http://forum.enjoysudoku.com/proposal-for-a-canonical-puzzle-t45209-15.html

but some quick answers

The name of the puzzle was first an attempt to reduce the size of the files. It appeared quickly that the canonical puzzle was a good tool to reach the target.
As you know, my canonical puzzle is the puzzle morphed to the minlex canonnical solulion grid (including auto morphs in the solution grid)

The "short name" is done of the solution grid rank plus a 81 bits field shown in text mode.
Up to now, I did not use the highest density of 19 bytes per sudoku. My tests show a limited reduction of the .zip files when I skip from 25 bytes to 19 bytes.
The 25 bytes name has the huge advantage to show clearly the solution grid.
BTW a slice of rank would be a good criteria to creat sub files. Puzzles sharing the same solution grid have many properties.

What is "clean expanded with subsets"?

For a player, if several puzzles have the same "hard step" (and the same solution grid) one is enough.

Using puzzles expanded with 'relatively easy step covered' is enough.

It appears that doing that, we reach expanded where one is a subset of the other.
Then, I keep only the puzzle with the smaller number of clues.
Small problem doing that, if you look for minimal of the final set of expanded, some puzzles of the original file are not there.