The New Sudoku Players' Forum

[champagne]

Hi coloin,
I have nearly finished the TH expansion with at the end 315333 expanded puzzles
I am checking that I have only puzzles with TH and I have possibly some without, likely coming from your files.

I have around 10K puzzles where my filter "suspicion of TH" gives nothing. I posted 2 in a separate thread where the problem is in my filter.

All puzzles of your post are rated skfr 11.8 or 11.7 on my side, but you know that I use it only as cutoff to select puzzles of interest.
I have now to check if I have all of them

On my side, I extrated the 11.9 seen in puzzles picked here and there (ED puzzle rated) and their expanded puzzles.
Funny but not very important for me, some of the expanded are rated 11.8 and not 11.9

here is my table with three solution grids hit

Code: Select all

......7.9.......3..89.....62...68..4.649.2..189.1......186.4...64..2....9.281....   898031988   05G6CckS1kO2T0

......7.9.......3..89.....62.1.68..4.649.2..189.14.....186.4...64..2....9.281....   898031988   05G6ickS3kO2T0   31   357   3


.....6.8..5.1..2.....3....5..5..386....5.81.28..26..53.4.......7.........38621...   2407740614   WI98aSeDp280{0
.....6.8..5.1..2...8.3....5..5..386....5.81.2...26..53.4.......7.........38621...   2407740614   WI9AaSe5p280{0
.....6.8..5.18.2.....3....5..5..386....5..1.28..26..53.4.......7.........38621...   2407740614   WIB8aS8Dp280{0
....56.8..5.1..2.....3....5..5..386......81.28..26..53.4.......7.........38621...   2407740614   mI98aSWDp280{0
....56.8..5.1..2...8.3....5..5..386......81.2...26..53.4.......7.........38621...   2407740614   mI9AaSW5p280{0

....56.8..5.18.2.....3.2..5..5.1386....5.81.2...26..53.4.......7.........38621...   2407740614   mIBeaUe5p280{0   30   479   7


.....6.8..5.1..2.....3....5..5..386....5.81.28...2..53.4.......7.........38261...   2407740662   WI98aSeDo280{0
.....6.8..5.1..2.....3.2..5..5.1386....5.81.28......53.4.......7.........38261...   2407740662   WI9eaUeDm280{0
.....6.8..5.1..2...8.3....5..5..386....5.81.2....2..53.4.......7.........38261...   2407740662   WI9AaSe5o280{0
.....6.8..5.1..2...8.3.2..5..5.1386....5.81.2.......53.4.......7.........38261...   2407740662   WI9gaUe5m280{0
.....6.8..5.18.2.....3....5..5..386....5..1.28...2..53.4.......7.........38261...   2407740662   WIB8aS8Do280{0
.....6.8..5.18.2.....3.2..5..5.1386....5..1.28......53.4.......7.........38261...   2407740662   WIBeaU8Dm280{0
....56.8..5.1..2.....3....5..5..386......81.28...2..53.4.......7.........38261...   2407740662   mI98aSWDo280{0
....56.8..5.1..2.....3.2..5..5.1386......81.28......53.4.......7.........38261...   2407740662   mI9eaUWDm280{0
....56.8..5.1..2...8.3....5..5..386......81.2....2..53.4.......7.........38261...   2407740662   mI9AaSW5o280{0
....56.8..5.1..2...8.3.2..5..5.1386......81.2.......53.4.......7.........38261...   2407740662   mI9gaUW5m280{0

.2..56.8..5.18.2.....3.2..5..5..386....5.81.2...6...53.4.......7.........38261...   2407740662   oIBeaSe5n280{0    29   479   7
....56.8..5.18.2.....3.2..5..5.1386....5.81.2...62..53.4.......7.........38261...   2407740662   mIBeaUe5p280{0   30   479   7


[coloin]

I have around 10K puzzles where my filter "suspicion of TH" gives nothing. I posted 2 in a separate thread where the problem is in my filter.

I saw them ... and nearly commented ...that you cant use elevens "3 clue substitution method" very easily on them !!!

Interesting ! that you have developed a tool which filters those not so straight forward tridagons too

I also thought that the {grid;pattern} [gp} labelling notation was rather apt ... much more apt than just "puzzle - name/19bytes plus zip" usage.
The name of a puzzle is more like "champagne dry" for example, and the puzzle label would be 5011249116;XIW4HXXY8W1JH0#

[champagne]

The run on all puzzles assumed to have a TH threat is finished; next step will be on the old PH2010 file not TH

I loaded to-day in my google drive a zip file name dy_ph_th_2510

d => database
y => “players”
ph => potential hardest
th => suspicion of th
2510 => yymm of release

This first database contains 315333 records in 147622 solution grids, all puzzles expanded till the point seen as “not too hard and quick to reach” in my code. For a player, this is a list covering all known puzzles with a TH rating skfr >=10.5

Here is a typical such record

Code: Select all

puz;rank;bitf;nclues;Er;digits;boxbase
.............8.1.37....2.4.2...78..4.7492..5.8.95.4..2.8724....5.2.974..94.8.5...;230208636;00gXAcUgbUeEh0;33;110;136;3



Note1: digits and box base refer to the TH assumed location. Digits=0 means that my code did not see where can be the TH.

Note 2: the current set of “quick” rules has no uniqueness code; This will change later and could lead to replace a record by another one being subset;

Note 3: Later, with new records coming from the vicinity search or of any other source, any record can be cleaned if a new subset is found.

Note 4: this set of puzzles is not enough to produce all known minimal puzzles used to build the expanded file see a post above

The file distribution by ER is

Code: Select all

Er   Count
105   7233
106   3602
107   334
108   8197
109   33995
110   54675
111   25685
112   8187
113   1682
114   7573
115   66700
116   83463
117   13847
118   159
119   1

[champagne]

Players file without TH threat

We start here with the old PH2010 file where mith did not see a TH.

The file has been adjusted to the skfr >=10.5 cutoff. This file is quite different from the TH file for this expansion project.

We have here 1 698 293 puzzles in 960 913 solution grids,

Many grids have a very low count

Code: Select all

npuz   num of solution grids
1   719428
2   132757


But at the other end, some grids have a very high count, here the list of grids with the highest count.

Code: Select all

3462750016   1061
147555454   873
2329807025   679
1751636880   666
2329805134   564
2353296059   495

I suspect that the puzzles in these grids have a TH not seen by mith, may be a degenerated one.
I had a look to the first one, the first puzzles entered the PH database in December 2011, when I started the search in the 26 clues field, and most of them came later posted by “Paquita”.

Also, in this project, looking for an expanded morph for solution grids with only one puzzle has no added value.

I need time to clarify several point .

EDIT a first check seems to show a complex degenerated TH threat in the first expanded puzzle with no subset

[coloin]

On my side, I extrated the 11.9 seen in puzzles picked here and there (ED puzzle rated) and their expanded puzzles.
Funny but not very important for me, some of the expanded are rated 11.8 and not 11.9

Yes there seems not a lot of difference at 11.8/11.9 level. FWIW here what I found

Hidden Text: Show

Code: Select all

Indeed the puzzle have different skfr on expansion. And SE has a reverse 11.8/11.9 of the ratings !!!
The difference between 11.8 and 11.9 is not very significant i guess.
And we know that there is different handling of the puzzle with isomorphic puzzles... and twin puzzles....

                                                                                    SE                skfr
......7.9.......3..89.....62.1.68..4.649.2..189.14.....186.4...64..2....9.281....#10#5# ED=11.8/11.8/3.4# ED=11.9/11.9/3.4#-1 

....56.8..5.18.2...8.3.2..5..5.1386....5.81.28..26..53.4.......7.........38621...# 6#2# ED=11.8/11.8/2.6# ED=11.9/11.9/2.6#-2A
....56.8..5.18.2.....3.2..5..5.1386....5.81.2...26..53.4.......7.........38621...

....56.8..5.18.2...8.3.2..5..5.1386....5.81.28..62..53.4.......7.........38261...# 6#2# ED=11.8/11.8/2.6# ED=11.9/11.9/2.6#-2B
....56.8..5.18.2.....3.2..5..5.1386....5.81.2...62..53.4.......7.........38261...

.2..56.8..5.18.2.....3.2..5..5..386....5.81.2...6...53.4.......7.........38261...# 8#1# ED=11.9/11.9/2.6# ED=11.8/11.8/2.6#-3

They give 24 minimal puzzles
......7.9.......3..89.....62...68..4.649.2..189.1......186.4...64..2....9.281....#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.1..2.....3....5..5..386....5.81.28..26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
....56.8..5.1..2.....3....5..5..386......81.28..26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.18.2.....3....5..5..386....5..1.28..26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.1..2...8.3....5..5..386....5.81.2...26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.18.2.....3....5..5..386....5.81.2...26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.8/1.2/1.2
....56.8..5.1..2...8.3....5..5..386......81.2...26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
....56.8..5.18.2.....3....5..5..386......81.2...26..53.4.......7.........38621...#      ED=11.8/1.2/1.2 # ED=11.8/1.2/1.2
.....6.8..5.1..2.....3....5..5..386....5.81.28...2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
....56.8..5.1..2.....3....5..5..386......81.28...2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.18.2.....3....5..5..386....5..1.28...2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.1..2...8.3....5..5..386....5.81.2....2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.18.2.....3....5..5..386....5.81.2....2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.8/1.2/1.2
....56.8..5.1..2...8.3....5..5..386......81.2....2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
....56.8..5.18.2.....3....5..5..386......81.2....2..53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.8/1.2/1.2
.....6.8..5.1..2.....3.2..5..5.1386....5.81.28......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
....56.8..5.1..2.....3.2..5..5.1386......81.28......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.18.2.....3.2..5..5.1386....5..1.28......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.1..2...8.3.2..5..5.1386....5.81.2.......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
.....6.8..5.18.2.....3.2..5..5.1386....5.81.2.......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.8/1.2/1.2
....56.8..5.1..2...8.3.2..5..5.1386......81.2.......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.9/1.2/1.2
....56.8..5.18.2.....3.2..5..5.1386......81.2.......53.4.......7.........38261...#      ED=11.8/1.2/1.2 # ED=11.8/1.2/1.2
.2...6.8..5.18.2.....3.2..5..5..386....5.81.2.......53.4.......7.........38261...#      ED=11.9/1.2/1.2 # ED=11.8/1.2/1.2
.2..56.8..5.18.2.....3.2..5..5..386......81.2.......53.4.......7.........38261...#      ED=11.9/1.2/1.2 # ED=11.8/1.2/1.2

which expand up to the 6 puzzles above using gsf -E

Code: Select all

1 has 5 twin puzzles 
......7.9.......3..89.....62.1.68..4.649.2..189.14.....186.4...64..2....9.281....#10#5#ED=11.8/11.8/3.4 # ED=11.9/11.9/3.4
...45.7.94..7.9.237...3254..6.......3.....95.5.1..........9437....27..95...3..4.2#10#5#ED=11.8/11.8/3.4 # ED=11.8/11.8/3.4
...45.7.94..7.9.237...3254..6.......3.....95.5.1..........7439....29..75...3..4.2#10#5#ED=11.8/11.8/3.4 # ED=11.8/11.8/3.4
......7.9.......3..89.....62.4.68..1.619.2..489.14.....186.4...64..2....9.281....#10#5#ED=11.8/11.8/3.4 # ED=11.9/11.9/3.4
.....6......78.....89.3.....94...31.31......58.5...9.45.1.4..93.38.1.45.94....8.1#10#5#ED=11.8/11.8/3.4 # ED=11.9/11.9/3.4

2A and 2B are twin puzzles
                                                                                        SE                skfr
....56.8..5.18.2...8.3.2..5..5.1386....5.81.28..26..53.4.......7.........38621...# 6#2# ED=11.8/11.8/2.6# ED=11.9/11.9/2.6#-2A
....56.8..5.18.2...8.3.2..5..5.1386....5.81.28..62..53.4.......7.........38261...# 6#2# ED=11.8/11.8/2.6# ED=11.9/11.9/2.6#-2B

2B and 3 share the same grid
.2..56.8..5.18.2.....3.2..5..5..386....5.81.2...6...53.4.......7.........38261...# 8#1# ED=11.9/11.9/2.6# ED=11.8/11.8/2.6#-3

.2..56.8..5.18.2...8.3.2..5..5.1386....5.81.28..62..53.4.......7.........38261...# 4#4# ED=10.6/1.2/1.2 # ED=10.4/1.2/1.2 # 2B and 3 amalgamated

2B and 3 amalgamated has 3 other twins , which provided a few more 11.8

.2..56.8..5.18.2...8.3.2..5..5.1386....5.81.28..62..53.4.......7.........38261...# 4#4# ED=10.6/1.2/1.2 # ED=10.4/1.2/1.2
.2..56.8..5.18.2...8.3.2..5..5.1386....5.81.28..26..53.4.......7.........38621...# 4#4# ED=10.6/1.2/1.2 # ED=10.4/1.2/1.2
.2345...945.7.9..37.9.23..52.......7..7....1....3726.457..3.....342.....9.2..7...# 4#4# ED=10.6/1.2/1.2 # ED=10.4/1.2/1.2
.234.6.8.4.678..2.78..23.6.23..7.....483.....6.7..8...3......7..7......1...83754.# 4#4# ED=10.6/1.2/1.2 # ED=10.4/1.2/1.2


....56.8..5.18.2...8.3.2..5..5.1386....5.81.28..26..53.4.......7.........38621...       ED=11.8/11.8/2.6# ED=11.9/11.9/2.6 -2A
.2..56.8..5.18.2...8.3.2..52.5.1386....5.81.28..26..53.4.......7.........38621...       ED=10.6/10.6/2.6# ED=10.4/10.4/2.6     
                                                                                                                               
....56.8..5.18.2.....3.2..5..5.1386....5.81.2...26..53.4.......7.........38621...       ED=11.8/11.8/2.6# ED=11.8/11.8/2.6     
.2..56.8..5.18.2.....3.2..52.5.1386....5.81.2...26..53.4.......7.........38621...       ED=10.6/10.6/2.6# ED=10.5/10.5/2.6

....56.8..5.18.2.....3.2..5..5.1386....5.81.2...62..53.4.......7.........38261...       ED=11.8/11.8/2.6# ED=11.8/11.8/2.6
....56.8..5.18.2...8.3.2..5..5.1386....5.81.28..62..53.4.......7.........38261...       ED=11.8/11.8/2.6# ED=11.9/11.9/2.6 -2B
                                                                                       
.2..56.8..5.18.2.....3.2..52.5.1386....5.81.2...62..53.4.......7.........38261...       ED=10.6/10.6/2.6# ED=10.5/10.5/2.6
.2..56.8..5.18.2...8.3.2..52.5.1386....5.81.28..62..53.4.......7.........38261...       ED=10.6/10.6/2.6# ED=10.4/10.4/2.6
                                                                                       
.2..56.8..5.18.2.....3.2..5..5..386....5.81.2...6...53.4.......7.........38261...       ED=11.9/11.9/2.6# ED=11.8/11.8/2.6 -3
.2..56.8..5.18.2...8.3.2..5..5..386....5.81.28..6...53.4.......7.........38261...       ED=11.7/11.7/2.6# ED=11.6/11.6/2.6
                                                                                       
.2345...945.7.9..37.9.23..52.......7..7....1.....726.457..3.....342.....9.2..7...       ED=11.8/11.8/2.6# ED=11.7/11.7/2.6       
.2345...945.7.9..37.9.23..52.......73.7....1....3726.457..3.....342.....9.2..7...       ED=10.6/10.6/2.6# ED=10.4/10.4/2.6

.234.6.8.4.678..2.78..23.6.23..7.....483.....6.7..8...3......7..7......1....3754.       ED=11.8/11.8/2.6# ED=11.7/11.7/2.6                                                                                       
.234.6.8.4.678..2.78..23.6.23..7.....483.....6.7..8...3......7.87......1...83754.       ED=10.6/10.6/2.6# ED=10.4/10.4/2.6

[champagne]

regarding puzzles with a high skfr rating, I think that the players are only interested by puzzles where they can shortly write

now easy to the end
now heavily downgraded
...

the rating as such is not so important, just a way to find puzzles to solve.

And surely with puzzles having a similar solving path at the hardest point, one is enough for them.

==================================
Having split the TH family, I have now a problem with the solution grids having plenty of puzzles with a high rating.
The first example shows a degenerated TH. I suspect that the same could be with solution grids not seen by mith as having a non degenerated TH.
I have no code to detect this degenerated TH and checking it without the computer is heavily time consuming.

In the first example, we have seeds coming out of the vicinity search done long ago and from the special work done by "paquita" later. I'll try to put such puzzles in a separate file.

I also intend to release quicly solution grids with one puzzle attached. Expansion here has no value and it is a big number of puzzles.
I'll see what to do with a low count of puzzles attached

And I have to finish this first, I have likely more than 2 millions puzzles rating >= skfr 10.5 waiting after a large vicinity done on the PH file

[champagne]

Thanks to “ghfik” for help, it appears that, in the residual solution grids, for the grid with 1061 puzzles having a high rating, the reason was a JE pattern, not a degenerated TH.

This open the door to some ideas of bridge JE  TH, but close the door to try here to send in the TH file some of the current puzzles of the "not TH file".

Consequently, I am nearly ready to release three files

dy_ph_th_2510 , already in my google drive, containing expanded puzzles for all puzzles assumed to have a non-degenerated TH

dy_ph_nth_single_2510 This file will contain 719428 puzzles of the old PH2010 having a solution grid with no other puzzle.

dy_ph_nthm_2510 for all other puzzles. This file should contain 550597 expanded puzzles.


At the end, this Potential Hardest for players status in 2025_10 contains

315333 expanded puzzles with the TH non-degenerated pattern seen by mith

And for puzzles not seen by mith as having a non-degenerated TH

719428 not expanded puzzles in solution grids with only one puzzle
550597 expanded puzzles

Th 2 missing files will be released in the next days; 

[champagne]

A short summary of the canonical name topic used in this thread.

The old PH had a “max lexical” morph of the puzzle as canonical name.
Here we use the min lexical morph of the solution grid in relay to open the door to very short names.

The topic is discussed in another thread
http://forum.enjoysudoku.com/proposal-for-a-canonical-puzzle-t45209.html

and the code to do it is in a github repository
https://github.com/GPenet/u06_naming

shortly

any sudoku has only one solution that can be morphed to the min lexical solution grid, one of the 5 472 730 538
(if the solution grid has auto morphs, we use the min lexical morph of the puzzle in it)

Then the puzzle can be shortly described as Solution grid rank plus 81 bits field of the cells given

Having the solution grid available is very important to detect similar puzzles and for many tasks.

Any bitfield can be shown in “ascii text mode” using 64 characters of the ascii table. Such a field is easy to use with many free tools working on text files. (See the relevant thread for more)

So my first canonical is the ED sudoku in 25 characters

12345.78..5......3...13.5..24.5.....3.129......5.4.2..5.231...4..49.5..29...24...;258945922;VJWOP1TWATabn0
10 for the rank left leading blanks _ One blank _ 14 charaters for the bitfield
giving this for file of names lenght 25

Code: Select all

  258945922 VJWOP1TWATabn0

For huge files, I use the highest density . The rank needs 33 bits, so the full description needs 33+81= 114 bits. This is 19x6bits

The code in the repository is not yet the final one. The main steps are available

Any sudoku to it’s ED morph in 25 bytes mode
Name 25 bytes to ED puzzle
Switch 25 => 19 and reverse.

Note: the “name” can only deliver the ED puzzle. The link to the “user morph” can only be done using more tables.

Later, I’ll use mainly the 25 bytes for a sudoku, and the 19 length for huge files, but as long as it will be possible, I keep the ED puzzle in released files to make it easier for users

[champagne]

No many new points here but interesting ones.

I entered in the drive a readme file to help users to use the files.

This readme gives the pointer to the naming code. This is currently a .exe program but a DLL will be ready next week for Linux users.

The start status is made of puzzles expanded and cleaned of {duplicates and puzzles having a subset}.
All puzzles of the old PH2010 and puzzles picked up here and there are used.
As written above, it is split in three files

Code: Select all

dy_ph_th_2510    for puzzles seen by “mith” as non-degenerated TH
dy_nth_singles_2510   for puzzles of the old PH2010 where the solution grid has only one puzzle
dy_ph_nthm_2510    for other puzzles of the old PH 2010


The 2 first bases are on the drive. The third is ready, but I am still working on it. What I found so far pushes me to release it soon.

As written earlier, it has 719 428 puzzles expanded in 550597 puzzles.
This is a poor ratio. The reason was a big surprise for me.

360696 puzzles could not be expanded using my process. The most surprising case was a solution grid with 49 puzzles not expanded. I’ll put this case in a separate thread to see what experts can say, but the common property of these puzzles is to have a very high skfr {EP rating }


Once this closed, I have millions of puzzles ready to be added. I have still some works to do to go in this direction. Among them:

My test to split “non degenerated TH” is not good, I want to keep in a separate file such puzzles very different.
Players want to have a flag “easy after special action” I am working to set it automatically in specific conditions. On example is puzzles with a “loki” style TH (only one guardian)

 

[champagne]

It seems that we are on the way to explain the first oddity seen in this work.

It is clear that a sudoku with a very high skfr EP rating will not expand in my process. The double JE is one pattern (may be the main one) appearing several times in some solution grids leading to a residual status of tens of sudokus similar for a player.
I’ll see later what can be done.

I have another oddity to understand.

In my database of expanded puzzles with the TH threat, for 10845 puzzles out of 315333, I could not find the TH digits and corner block using my simplified process.
I changed my code looking now for the 3 digits in the corner box, and it turned to 14278 puzzles missing the standard this in 12415 different solution grids. Most of the puzzles are common to both sets

Usually, the solution grid has no other expanded except for 132 of them.

This is worth explaining it before the addition of new puzzles. I’ll start the addition only for puzzles having the TH suspicion quickly seen.

I posted one example of 2 solution grids showing this oddity.
As far as I can see, the resolution path could be very surprising

[champagne]

Relying on the results of the example posted, I'll take all these solution grids as having no TH.
All these puzzles come from what I picked up in the former thread or received.

I have to code a full recognition of the TH pattern as soon as possible, meantime, I keep the released files as they are

In the next release, all puzzles of these grids will be moved to the "noTH" file(s).
Also, in the nest release, the link to the old PH2010 will be cut

The design of the TH file will slightly change with flags telling if some clearing is possible and if the puzzles becomes easy after
The "noTH" file will loose data from the old PH2210 : source gird, and references to the entry. New flags will be added as in the TH file.

[champagne]

I finished the first phase, moving part of the puzzles from TH to non TH

At the end, I have

135273 solution grids hit in "assumed TH"
250678 solution grids hit in "assumed non TH"

the 2 sets are disjoints except for 454 grid.

Nothing object to having in the same solution grid both pattern with high SKFR ratings.
But my priority is now to add several millions of TH puzzles waiting to be merged. This will take several days, checking carefully that nothing is missing.

here are some of the solution grids common to both files

Code: Select all

200719884   208434679   216942133   429497843
452307986   603319287   607395124   682970527
757580026   844489012   849772743   850190807
890647636   890648669   905591826   917869728
917870537   938830677   940013939   940013941
953523163   1018220984   1021420820   1029356524


[mith]

I had already morphed mith’s file and my old “PH” file to the canonical morph and re rated all, using SKFR, So, the big task has been to pick up all puzzles seen here and there or got by other ways.

Surely, some puzzles have been lost, but I had at the end 1 180 459 canonical puzzles not in the 2 previous files, mostly rating >=11.5, some minimal, some not.

The second step has been to find all “minimal” and to check again what was new against the 2 files.

I have now 4 267 978 new minimal puzzles in hand. As most of them have a skfr rating >11.5; re rating this file will take several days. As the suppliers use often skfr or serate as pre filter, only a small number of them will be cleaned with the cutoff of 10.5. As a side effect, the skfr rating seems to be mainly in the 4/5 seconds per puzzle, a high value for millions of puzzles.

I know this is a month late at this point, but the 4.6+ million minimals in my database as of August 2023 were posted here: post338328.html#p338328 (the post mentions updating credit for puzzles posted prior to this link, but I never got to that... got bogged down in trying to set up some tools for parsing the forum posts and extracting credit that way) I hadn't done a full update with the expanded database (and min-expands, max-expands) so your 1.18 million puzzles were likely missing some from there.

If you don't need to re-rate them in your canonical form, and haven't already done the re-rating, this should save you some time!

(Still not back to a point where I can get things up and running again, but I'm getting closer to tinkering with things. I'll try to at least get a snapshot of the state of my databases when last I worked on this, in the meantime. The number of minimals I have is more like 36 million, though not all of those are worth considering (lots in the high 10/low 11 SER range).

[mith]

I can also share my trivalue oddagon search algorithm with you at some point if you like. It can identify all non-degenerate [my defintion] TOs for the puzzles in the T&E(3) databases - results for min-expands and max-expands as of November 2022 are in the second post of the T&E(3) thread.

[champagne]

Hi mith,

I did my job on your 4 millions T&E3 file

swith to canonical ED morph
re rating
cleaning of puzzles solved easily using UR

This is what has been expanded and reduce to "no subset"

Your 36 million minimal should not be lost. If I want to work on it

a first unavoidable step is to switch to the canonical ED solution grid.
Usuallly, only a small redundancy can come here due to grids auto morphs.

At this point the file can be checked for cleaning with all minimal known on my side.

I assume that all of them have the TH pattern, so we can expect a significant reduction using expanded puzzles.
I would do it, than check again for redundancy and rate (SKFR) this reduced file. With the cutoff of 10.5, all minimal attached to an expanded below the cutoff can be cleaned.

Big task, but nothing harder than what I am doing to-day

=================================== TH recognition

This is not a key point for the players database(s), but
the TH pattern delivers usually many puzzles per solution grid with a small number of expanded puzzles
It is a pattern with a specific logic

so I think of keeping such puzzles in a separate database with appropriate flags.

And yes, I have no reliable quick recognition of the TH threat. I used as much as possible your work to send puzzles in the appropriate file.

My current quick recgnition sees as "TH threat" all puzzles having 3 digits with less than 2 clues and all clues in the same box.

I followed your links but did not see what is your recognition process.
I can surely code a full analysis, but this take times (I did it on solution grids to see where we had a potential target for a non degenerated TH). I am interested in seeing what you did.

=================== history of the TH

In the second post of this thread, I started an history of the TH using your analysis of the PH2010 file.

I see more and more some similarities exocet<=> TH, but you surely are the best to say if this is true and if the TH came from seeds taken in the PH database