The New Sudoku Players' Forum

[champagne]

Mith's file reshaping is slower than I hoped, and will be covered in 3/4 days.
An interesting partial result.

This is for about 2.7 million puzzles of the file.
I find a significant redundancy, (likely more than 1000 puzzles at the end) .
I assume that this comes from auto morphs in the solution grids.

More interesting is the new ER (skfr rating) distribution

in the table, the first column is the ER rating x10 so 45 is skfr 4.5
The second column is the number of puzzles in the data base havbing this rating
The third column is a progressive sum of the second colum.

Code: Select all

 45   1963   1963
46   12   1975
47   1   1976
56   68   2044
66   2472   4516
67   135   4651
68   3   4654
71   675   5329
72   88   5417
73   26   5443
76   136   5579
77   72   5651
78   145   5796
79   2   5798
80   14   5812
83   553   6365
84   374   6739
85   177   6916
87   6   6922
88   18   6940
89   655   7595
90   8277   15872
91   10647   26519
92   32549   59068
93   5585   64653
94   288   64941
95   481   65422
96   605   66027
97   988   67015
98   239   67254
99   910   68164
100   60200   128364
101   219650   348014
102   661611   1009625
103   326249   1335874
104   236870   1572744
105   61154   1633898
106   12134   1646032
107   1143   1647175
108   105172   1752347
109   288256   2040603
110   65629   2106232
111   84864   2191096
112   3822   2194918
113   773   2195691
114   2455   2198146
115   7388   2205534
116   360833   2566367
117   126500   2692867
118   70   2692937

If I use the same cutoff as in my current work, ER 105, 1 572 744 puzzles (58% of the total) are ignored.
I assume that most of them are downgraded due to the uniqueness rules, but this is in line with my choice to ignore puzzles easy to solve by a player.

So far, the max ER seen by skfr is 11.8, same as in my current work, and the number of 11.8 is very small.

I wait for the final result for more comments, but if this reflects the final status, my new high ratings are very similar to this distribution.

[coloin]

I find a significant redundancy, (likely more than 1000 puzzles at the end) ........

Yes ... I did clean up his file as there were a only few duplicates and a few non minimal puzzles.....
I min expanded and reminimized and only a few more were found... so mith already had done this process.

As I said its much more efficient to min-expand the puzzles ... and then do the rating/analysing.

these 24 minimal puzzles [of yours recently posted] expand to 4 min-expanded puzzles

Hidden Text: Show

Code: Select all

....567...567..1...8.1.3..........4.3.......2...83.5..51.6.8...6...7.....37.15...;11.8/1.2/1.2
....567...567..1...8.1.3.........34.3.......2...8..5..51.6.8...6...7.....37.15...;11.8/1.2/1.2
..3.567...567..1...8.1.3..........4.3.......2...83.5..51.6.8...6..37......7.15...;11.8/1.2/1.2
..3.567...567..1...8.1.3.........34.........2...83.5..51.6.8...6...7.....37.15...;11.8/1.2/1.2
..3.567...567..1...8.1.3.........34.3.......2...8..5..51.6.8...6..37......7.15...;11.8/1.2/1.2
.2.4.6............789........53..87...8....92......3.53.2.7.95...7.3.2.8.9......7;11.8/1.2/1.2
.2.4.6............789........53..87...8....92.7....3.53.2.7.95.....3.2.8.9......7;11.8/1.2/1.2
.2.4.6............789........53..87..38....92........5..2.7.95...7.3.2.8.9.....37;11.8/1.2/1.2
.2.4.6............789........53..87..38....92.7......5..2.7.95.....3.2.8.9.....37;11.8/1.2/1.2
.2.4.67...........789........53..8....8....92.7....3.53.2.7.95.....3.2.8.9......7;11.8/1.2/1.2
.2.4.67...........789........53..8....8....92.7....3.53.2.7.95...7.3.2.8.9.....3.;11.8/1.2/1.2
.2.4.67...........789........53..8...38....92.7......5..2.7.95.....3.2.8.9.....37;11.8/1.2/1.2
.2.4.67...........789........53..8...38....92.7......5..2.7.95...7.3.2.8.9.....3.;11.8/1.2/1.2
.2.4.67...........789........53..87...8....92......3.53.2.7.95...7.3.2.8.9.......;11.8/1.2/1.2
.2.4.67...........789........53..87...8....92.7....3.53.2.7.95.....3.2.8.9.......;11.8/1.2/1.2
.2.4.67...........789........53..87..38....92........5..2.7.95...7.3.2.8.9.....3.;11.8/1.2/1.2
.2.4.67...........789........53..87..38....92.7......5..2.7.95.....3.2.8.9.....3.;11.8/1.2/1.2
1...567...567..1...8...3..........4.3.......2...83.5..51.6.8...6...7.....37.15...;11.8/1.2/1.2
1...567...567..1...8...3.........34.3.......2...8..5..51.6.8...6...7.....37.15...;11.8/1.2/1.2
1...567...567..1..78...3..........4.3.......2...83.5..57.6.8...6...1.....31.75...;11.8/1.2/1.2
1.3.567...567..1...8...3..........4.3.......2...83.5..51.6.8...6..37......7.15...;11.8/1.2/1.2
1.3.567...567..1...8...3.........34.........2...83.5..51.6.8...6...7.....37.15...;11.8/1.2/1.2
1.3.567...567..1...8...3.........34.3.......2...8..5..51.6.8...6..37......7.15...;11.8/1.2/1.2
1.3.567...567..1..78...3..........4.3.......2...83.5..57.6.8...6..31......1.75...;11.8/1.2/1.2
                                                                                             
1.3.567...5678.1..78.1.3..........4.3.......2...83.5..57.6.8...6..31.....31.75... BxB 10                 
1.3.567...5678.1..78.1.3..........4.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10                 
1.3.567...5678.1..78.1.3.........34.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10      ###
.2.4.67...........789........53..87..38....92.7....3.53.2.7.95...7.3.2.889.....37 BxB 5                 
                                                                                                         
The  ### puzzle has min expanded maximally to the "max-expand" and has 3 other twin puzzles             
1.3.567...5678.1..78.1.3.........34.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10 ED 11.8/11.8/3.4
1.3.567...5678.1..78.1.3.........34.3.......2...83.5..57.6.8...6..31.....31.75... BxB 10 ED 11.8/11.8/3.4
.234.6.......8.........1...23..6..47.67...2..9.4...63.3.96...7267....49..429..3.6 BxB 10 ED 11.8/11.8/3.4
........9.56...13........4..376.....5.83.1...61..75..337.1.8...8.5.673...6153.8.. BxB 10 ED 11.8/11.8/3.4
........9.56...13........4..376.....61..75..38.53.1...37.1.8...5.8.673...6153.8.. BxB 10 ED 11.8/11.8/3.4



so instead of 4 million minimals there are 300000 min-expands ... which is more manageable [for me]

[champagne]

I did not started an intensive use of the min expand, but I follow you.
To-day, i have the first steps of the vicinity on my seeds coming out of the direct search.
My priority is to understand what happens and to build the process to use later.

As you, I now do the vicinity only on minimal puzzles, so my filters to limit redundancy are also on minimal puzzles.
I know that the {expand / get minimals} cycles are efficient, but I had no time so far to launch them.


I had a look to your list,

Code: Select all

1.3.567...5678.1..78.1.3..........4.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10                 
1.3.567...5678.1..78.1.3.........34.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10      ###

the second puzzle seems to be an expand of the first
and I have difficulties to follow your "twin puzzles" chapter, but I am not familiar with the process.

[coloin]

I think you have to do the vicinity on minimal puzzles... otherwise too many are produced.
The expansion function -E ... is consistent and I cant see it affecting the rating of all minimal puzzles which expand to that min-expand.
Yes these were the min expands of your puzzles, one didnt expand as much. and the other obvious twin didn't max-expand.

The max expand process is longish to compute. Expansion to keep SKFR and BXB are similar but not identical. The more clues you add the more minimal puzzles and twins can be made with that rating.
Expanding the puzzle [keeping the solution grid] could likely be performed with an internal {-2+2} .. but you do seem to find "all" the minimals in any one grid.
A max expand puzzle could be created by "summating" all the clues in all the puzzles which rate at the high level .... this occured with your puzzles...
occasionally this max-expand has a reduced rating, but this could be checked and happens very rarely.... but the process would be considerably quicker.

The [max-expand] twin puzzles might well give you a means to expand the vicinity search more than {-1+1], and the resultant twin puzzles do minimize differently too.
all done with gridchecker command line
gridchecker --similar --twins <test.txt > testtwins.txt
It does demand a clean file without gaps or spaces, output is in the minlex puzzle which needs changing to minlexgrid puzzle.

[Hajime]

I had a look to your list,

Code: Select all

1.3.567...5678.1..78.1.3..........4.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10                 
1.3.567...5678.1..78.1.3.........34.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10      ###

the second puzzle seems to be an expand of the first

Both puzzles are in T&E(2) and not in T&E(3), are they? So no trigadon puzzle with SER=11.8 ?

[champagne]

I had a look to your list,

Code: Select all

1.3.567...5678.1..78.1.3..........4.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10                 
1.3.567...5678.1..78.1.3.........34.3.......2...83.5..51.6.8...6..37.....37.15... BxB 10      ###

the second puzzle seems to be an expand of the first

Both puzzles are in T&E(2) and not in T&E(3), are they? So no trigadon puzzle with SER=11.8 ?

I can not say if this is T&E2 or T&E3, I am only working on the skfr rating
But I don't understand your last point. I posted plenty of puzzles with a tridagon rating skfr 11.8, and likely some of them rating SER 11.9

I have nearly finished the re rating of mith's file where many puzzles have a rating SER 11.9, so far, I only had a rating skfr 11.8

To be honest, I don't think that the rating before the tridagon analysis is so important. It remains for me a filter to select puzzles of interest.

[Hajime]

Hi champagne,
I am not looking for tridagons either. But I am interested in Sudoku's with a very high rating (skfr or SER)
Tridagons were discovered in 2022 or so. Loki and Thor's Hammer are tridagons that could not not be solved in T&E(2).
I think all tridagons are in T&E(3) , but there are more (non-tradagons) in T&E(3). I did not check.

Experienced human solvers can solve tridagons, and if a method was implemented in skfr or SE, they would not have such a high rating.
So puzzles with a high rating but not a tridagon are of my interest.

[champagne]

Hi champagne,
I am not looking for tridagons either. But I am interested in Sudoku's with a very high rating (skfr or SER)
Tridagons were discovered in 2022 or so. Loki and Thor's Hammer are tridagons that could not not be solved in T&E(2).
I think all tridagons are in T&E(3) , but there are more (non-tradagons) in T&E(3). I did not check.

Experienced human solvers can solve tridagons, and if a method was implemented in skfr or SE, they would not have such a high rating.
So puzzles with a high rating but not a tridagon are of my interest.

years ago, I worked hard on puzzles with a high SER rating. I updated a data base of such puzzles.
I switched to other topics in 2010, and as far as I know, nobody took over the job.

Many of these puzzles had an "exotic pattern" with a first potential step very similar to what appears with the tridagon/TH.
If you know the properties of the pattern, then you can reduce the puzzle to something relatively easy.

When "loki" and the TH came, most people working in this field turned, as far as I know, in the search of such puzzles.
I as well was curious to see how common this pattern could be and I am working for nearly six months on this topic.

As I improved, in this work, my vicinity search, I could try later to see what can be done around known high ratings.

I think that I have still 2/3 months to work on the tridagon pattern.

[champagne]

I have finished the reshaping, except that I lost the last chunk, about 32K puzzles. I’ll run it later, not a problem to look at the results.

From the 4 634 322 records, 4 600 000 have been reshaped giving 4 598 631 ED puzzles

The first action has been to erase all puzzles rating below skfr 10.5, to be in line with my cutoff.

As already seen, more than 50% of the puzzles are cleared and the file has now 1 931 510 puzzles.

The count per skfr/ER is the following

Code: Select all

105   112803
106   20442
107   1443
108   177704
109   502806
110   111176
111   145275
112   6135
113   1509
114   4332
115   13078
116   601970
117   232706
118   131

Most of the potential best puzzles have a rating 11.6 11.7.

We had more than 60K solution grids in mith’ file, we have here 40 652 solution grids.

“loki” is in the grid rank 3636310645. This grid has only four puzzles in this reduced mith’s file, all rating skfr 11.8
===================================================
The amazing case is for the solution grid 3967132101

This grid has 11019 puzzles in mith’s file rating >= 10.5 and all of them are rating 11.7

Here the first 2 puzzles in my data base
.2..56......18.......7.2..528...7...319...5...7.....6.......85.8...7.6.1..5..1.27
.2..56......18.......7.2..5286......31.....7.5.4....6.......8..8...756.1..5..1.27
=============================================
Many solution grids have a low count of puzzles

Count Ofpuz Compte of solution grids

Code: Select all

1   1231
2   2827
3   1467
4   3322
5   918
6   2435

And on the other side of the table, with one solution grid in each case we have the counts

2045 2289 2331 2338 2475 2531 3310 3316 3539 11019

In this minlex morph, we have 650 different magic squares.
This is not far from the maximum 9*84. A deeper analysis (location of the magic cells in the solution grid) is not easy to do and would be of small interest IMO.

We are missing the history of this file, but it’s hard to believe that this is just the result of a vicinity starting from “loki considering the low count of puzzles in “loki”’s solution grid.

On my side,
my first family, “kuto” has a start very similar to the “loki” solution grid and seems to have a very limited potential through the vicinity.
And I have a start with respectively 1117 929 733 668 puzzles for the best ones;

It will be interesting to see what happens in the first cycles.

[coloin]

As expected.... all the 11019 minimal puzzles come from this "max-expand" with 39 clues

Code: Select all

+---+---+---+
|.2.|.56|7..|
|.5.|18.|2.6|
|...|7.2|..5|
+---+---+---+
|286|5.7|...|
|319|...|57.|
|574|...|.6.|
+---+---+---+
|...|...|85.|
|8.2|.75|6.1|
|..5|8.1|.27|
+---+---+---+    39C


the minimal puzzles "min-expand" to only these puzzles

Code: Select all

.2..567.....18.2.6...7.2..52....7...319....7.574....6.......85...2..56.1...8.1.27
.2..567.....18.2.6...7.2..52....7...319....7.574....6.......85...2.756.1...8.1.27
.2..567.....18.2.6...7.2..52....7...319....7.574....6.......85.8.2.756.1...8.1.27
.2..567.....18.2.6...7.2..52..5.7...319...57.574....6.......85...2..56.1...8.1.27
.2..567.....18.2.6...7.2..52..5.7...319...57.574....6.......85...2.756.1...8.1.27
.2..567.....18.2.6...7.2..52..5.7...319...57.574....6.......85.8.2.756.1...8.1.27
.2..567.....18.2.6...7.2..528...7...3.9....7.574....6.......85.8.2.756.1...8.1.27
.2..567.....18.2.6...7.2..528.5.7...3.9...57.574....6.......85.8.2.756.1...8.1.27
.2..567.....18.2.6...7.2..5286..7...319....7.574....6.......85.8.2.756.1...8.1.27
.2..567.....18.2.6...7.2..52865.7...319...57.574....6.......85.8.2.756.1...8.1.27
.2..567...5.18.2.6...7.2..52....7...319....7.574....6.......85...2..56.1..58.1.27
.2..567...5.18.2.6...7.2..52....7...319....7.574....6.......85...2.756.1..58.1.27
.2..567...5.18.2.6...7.2..52....7...319....7.574....6.......85.8.2.756.1..58.1.27
.2..567...5.18.2.6...7.2..52..5.7...319...57.574....6.......85...2..56.1..58.1.27
.2..567...5.18.2.6...7.2..52..5.7...319...57.574....6.......85...2.756.1..58.1.27
.2..567...5.18.2.6...7.2..52..5.7...319...57.574....6.......85.8.2.756.1..58.1.27
.2..567...5.18.2.6...7.2..528...7...3.9....7.574....6.......8..8.2.756.1...8.1.27
.2..567...5.18.2.6...7.2..528...7...3.9....7.574....6.......85.8.2.756.1..58.1.27
.2..567...5.18.2.6...7.2..528.5.7....19...57.574....6.......85.8.2.756.1..58.1.27
.2..567...5.18.2.6...7.2..528.5.7...3.9...57.574....6.......8..8.2.756.1...8.1..7
.2..567...5.18.2.6...7.2..528.5.7...3.9...57.574....6.......85.8.2.756.1..58.1.27
.2..567...5.18.2.6...7.2..5286..7...319....7.574....6.......85.8.2.756.1..58.1.27
.2..567...5.18.2.6...7.2..52865.7...319...57.574....6.......85.8.2.756.1..58.1.27    [39C]


I save a lot of time by converting to these min-expands .. as all the minimals expand to only these ones..

[coloin]

In this minlex morph, we have 650 different magic squares.
This is not far from the maximum 9*84. A deeper analysis (location of the magic cells in the solution grid) is not easy to do and would be of small interest IMO.

What exactly is a magic square ? !

[champagne]

I save a lot of time by converting to these min-expands .. as all the minimals expand to only these ones..

First, good to have the check for the max expand. But it remains very interesting to know that we have so many ways to build the tridagon threat and this gives more vicinity potential. As you wrote elsewhere, the vicinity is usually done on minimal puzzles.


And I use commonly the "magic square" term to qualify the 4 boxes pattern where is the tridagon.

[champagne]

But your example convinced me to put on the top of the to do list the expand/maxexpand topic.
Having the data base of puzzles in canonical morph available, I have ideas on something not to hard to implement