To avoid endless and fruitless discussions, I open a new thread to collect some results on ratings below the potential hardest.
It’s clear that an intensive work has been done over time to find and analyse the hardest puzzles.
There were two main reasons to focus on such puzzles:
- People like challenging targets,
- There was no volume issue for such puzzles.
It appeared quickly that puzzles defined as “potential hardest” had specific properties giving sometimes a short quick and easy solution.
The Exocet pattern can be found in more than 75% the puzzles and in total, more than 80% have one of the “exotic” properties.
For several reasons, the main one being the volume issue, no significant collection of puzzles existed for lower ratings.
Recently the question of the frequency of the “exotic” patterns for lower ratings has been raised, it appeared that we were lacking information to even try to answer to that question.
On the other side, some players were interested in getting some puzzles offering some of the “exotic” patterns with lower ratings, and, if possible, patterns differing from the ones found for the potential hardest file.
At the end, 2 actions were started.
a) Eleven tried produced a “pseudo random” sample of puzzles. As the price to pay to have puzzles with significant ratings is very high, the file he got remains in the lowest part of the ratings range to study.
b) I launched a vicinity +-3 search on the potential hardest file keeping all puzzles generated.
The corresponding sample file will be the basis for deeper investigations described in that thread.
Eleven’s file has 282588 puzzles rating (skfr rating) 8.6 to 9.3, with 2380 puzzles (less than 1% of the sample) rating respectively 9.1 (2219) 9.2 (158) and 9.3 (3);
The vicinity search produced 209.7 millions puzzles not in the data base of potential hardest rating more than 7.5 and 56.9 million puzzles rating between 6.2 and 7.5
One interesting result is the ratio JExocet/Number of puzzles in Eleven’s sample.
The ratio is 0.03% (raw figures an adjusted 0.018% has been worked out, but this does not change the scope).
In the file of potential hardest, the same ratio is 76%.
If we assume that the distribution of that ratio is not linked to the rating, we can expect the same ratio in the ratings over 10.3 (the entire file of potential hardest) than in eleven’s sample
Then, we should expect the entire file of ratings over 10.3 to be at least 4000 times bigger than the current file.
Everybody having worked in that field will tell that this is crazy. The best we can expect is to have the entire file with clues in the range 20-25 to be 2 to 10 times bigger than the current file.
Then, we must assume a ratio JE/number of puzzles depending on the rating and we come to eleven’s assumption that the ratio will sharply decline from the highest level to the sample area (8.6 to 9.0)
The sample coming out of the vicinity search should give a upper limit of that tendency. The ongoing rating of the entire sample is a necessary step to have that upper limit.
Next posts will describe in details the results of tests done on the 2 samples and where to find the corresponding data.
Regarding the JE (and other Exocets) frequency, I decided to keep here the non truncated process.
Additional selections will follow to meet David’s expectations. This will give more homogeneity in the data sets.
storing the results
we have to share huge volumes, which is not easy.
Some of the results will be located in the skfr project where we have an allocation of 4 GB
Google does not allocate storage place for new projects, so I'll use the new allocation, something as storage in the cloud to locate the raw results of the samples.
the link to that storage place is here
having no experience of that new tool, I started with a readme file and the list of puzzles in eleven sample having a rank 0 logic seen by my solver;
pieces of the big sample should come after the rating will be over
exotic patterns below the potential hardest
27 posts
• Page 1 of 2 • 1, 2
exotic patterns below the potential hardest
by champagne » Sat Jun 22, 2013 5:44 am
Last edited by champagne on Mon Jun 24, 2013 12:17 pm, edited 1 time in total.
- champagne
- 2017 Supporter
- Posts: 7465
- Joined: 02 August 2007
- Location: France Brittany
eleven'sSample file below potential hardest
by champagne » Sat Jun 22, 2013 5:45 am
Eleven sample has been shown first in the thread JExocet Pattern Defintion Page 13
Here are pieces of posts related to that sample
From eleven presentation
Now I stopped the generation of pseudo random puzzles with 86 mio puzzles.
I uploaded all puzzles with skfr >=8.6 here (7MB) and about 0.5 mio with skfr between 7.5 and 8.5 here (about 15MB).
This is the ratings distribution > skfr 6.1 in absolute numbers found in 86 mio puzzles.
I focused on the file with skfr rating >= 8.6. Here after, I am only referring to that one.
The clues distribution in that file is the following
The first study on that file (I voluntarily ignore here the lowest ratings) is related to the Exocet frequency for such ratings.
Here we face a first problem that we can have for any exotic pattern.
With potential hardest, puzzles, nearly nothing could be done at start with usual rules. The common way to detect an exotic pattern has been to solve up to a certain point and then to look for the exotic pattern.
With lower ratings, the situation seems more complex. Depending on what is done at the start, we can have different results.
My first search for JE’s in the file found 75 puzzles with one or more JE’s
I changed the actions at start and found respectively 71,19 and 47 puzzles in three different runs, with a combined total of 90 puzzles
“Blue” made a deeper search shown here
I keep from that table only 3 and 4 digits and I omit “box extension” not implemented in my process.
A) Standard JExocet (with the allowances for "degenerate cases")
G) Base cells in a mini-row/col, target cells in the base band/stack, targets in different boxes.
H) Base cells in a mini-row/col, target cells in the base band/stack, targets the same box.
K) Base cells in a mini-row/col, at least one target outside the base band/stack.
Note 1: blue counts the number of Exocets, I count the number of puzzles having the property in the run
I looked for exocets non JE and got the following equivalent results :
(after my standard eliminations with, here, a minimum of 40 cells unknown)
A+G+H 9439 puzzles
A+H+H+K 11600 puzzles
Knowing that blue has seen many exocets in some of the puzzles, we have the same tendency and I share blue’s remark
Also interesting: compared with champagne's "potential hardest" list, these puzzles have a much larger ratio of general exocets to "JExocet-like" exocets.
Regarding other exocet patterns I found
0 SK loop
2260 puzzles containing a rank 0 logic.
Having no sk loop is not astonishing. The sk loop appears with specific patterns. Having an overall low frequency, even in the potential hardest file and being focused in specific patterns, it had small chances to come in eleven’s sample.
The rank 0 logic ratio is not bad (close to 1% ). Knowing that my program detects only part of them, it remains an interesting solving technique in that area.
I’ll make in a separate post a typology of the corresponding puzzles.
Here below the distribution of the 2260 puzzles found with a rank 0 logic
I changed the code extending the search of rank 0 logic (mainly to the 2 digits pattern)
instead of 2260 , I found 6943 puzzles having a rank 0 logic.
Subject to more checking, this is not so far from the frequency seen in the potential hardest area.
Here are pieces of posts related to that sample
From eleven presentation
Now I stopped the generation of pseudo random puzzles with 86 mio puzzles.
I uploaded all puzzles with skfr >=8.6 here (7MB) and about 0.5 mio with skfr between 7.5 and 8.5 here (about 15MB).
This is the ratings distribution > skfr 6.1 in absolute numbers found in 86 mio puzzles.
Hidden Text: Show
I focused on the file with skfr rating >= 8.6. Here after, I am only referring to that one.
The clues distribution in that file is the following
Hidden Text: Show
The first study on that file (I voluntarily ignore here the lowest ratings) is related to the Exocet frequency for such ratings.
Here we face a first problem that we can have for any exotic pattern.
With potential hardest, puzzles, nearly nothing could be done at start with usual rules. The common way to detect an exotic pattern has been to solve up to a certain point and then to look for the exotic pattern.
With lower ratings, the situation seems more complex. Depending on what is done at the start, we can have different results.
My first search for JE’s in the file found 75 puzzles with one or more JE’s
I changed the actions at start and found respectively 71,19 and 47 puzzles in three different runs, with a combined total of 90 puzzles
“Blue” made a deeper search shown here
I keep from that table only 3 and 4 digits and I omit “box extension” not implemented in my process.
- Code: Select all
282588 puzzles
+--------------+
| 3 4 |
+---+--------------+
| A | 104 25 |
+---+--------------+
| G | 2419 1219 |
| H | 8899 4328 |
| K | 19237 11012 |
+---+--------------+
A) Standard JExocet (with the allowances for "degenerate cases")
G) Base cells in a mini-row/col, target cells in the base band/stack, targets in different boxes.
H) Base cells in a mini-row/col, target cells in the base band/stack, targets the same box.
K) Base cells in a mini-row/col, at least one target outside the base band/stack.
Note 1: blue counts the number of Exocets, I count the number of puzzles having the property in the run
I looked for exocets non JE and got the following equivalent results :
(after my standard eliminations with, here, a minimum of 40 cells unknown)
A+G+H 9439 puzzles
A+H+H+K 11600 puzzles
Knowing that blue has seen many exocets in some of the puzzles, we have the same tendency and I share blue’s remark
Also interesting: compared with champagne's "potential hardest" list, these puzzles have a much larger ratio of general exocets to "JExocet-like" exocets.
Regarding other exocet patterns I found
0 SK loop
2260 puzzles containing a rank 0 logic.
Having no sk loop is not astonishing. The sk loop appears with specific patterns. Having an overall low frequency, even in the potential hardest file and being focused in specific patterns, it had small chances to come in eleven’s sample.
The rank 0 logic ratio is not bad (close to 1% ). Knowing that my program detects only part of them, it remains an interesting solving technique in that area.
I’ll make in a separate post a typology of the corresponding puzzles.
Here below the distribution of the 2260 puzzles found with a rank 0 logic
Hidden Text: Show
I changed the code extending the search of rank 0 logic (mainly to the 2 digits pattern)
instead of 2260 , I found 6943 puzzles having a rank 0 logic.
Subject to more checking, this is not so far from the frequency seen in the potential hardest area.
Last edited by champagne on Mon Jun 24, 2013 7:39 am, edited 4 times in total.
- champagne
- 2017 Supporter
- Posts: 7465
- Joined: 02 August 2007
- Location: France Brittany
Re: exotic patterns below the potential hardest
by ronk » Sat Jun 22, 2013 9:54 pm
champagne & moderators, the subject of the distribution of a sudoku pattern, exotic or not, certainly does not qualify as an advanced solving technique. Such threads belong in the General sub-forum IMO, not Advanced solving techniques.
- ronk
- 2012 Supporter
- Posts: 4764
- Joined: 02 November 2005
- Location: Southeastern USA
Re: exotic patterns below the potential hardest
by champagne » Sun Jun 23, 2013 8:18 am
here is a file extracted from eleven's sample
my solver found a rank 0 logic in them and the count of truths is very low.
I kept puzzles with <=8 truths, knowing that my solver ignores solutions with less than 6 truths and that the search here did not consider a 2 digits logic
the file contains the following data
puzzle
truths count
number of digits
rating
digits used to define the rank 0 logic
rank 0 logic type (1 rows 2 columns 4 "X")
I did not dig in the file, but we have at the end some puzzles with 8 truths and 5 digits; I am sure they can be converted in a logic with a lower number of digits.
I intend to restart the study in the 2 digits field
my solver found a rank 0 logic in them and the count of truths is very low.
I kept puzzles with <=8 truths, knowing that my solver ignores solutions with less than 6 truths and that the search here did not consider a 2 digits logic
the file contains the following data
puzzle
truths count
number of digits
rating
digits used to define the rank 0 logic
rank 0 logic type (1 rows 2 columns 4 "X")
I did not dig in the file, but we have at the end some puzzles with 8 truths and 5 digits; I am sure they can be converted in a logic with a lower number of digits.
I intend to restart the study in the 2 digits field
Hidden Text: Show
- champagne
- 2017 Supporter
- Posts: 7465
- Joined: 02 August 2007
- Location: France Brittany
Re: exotic patterns below the potential hardest
by champagne » Mon Jun 24, 2013 7:43 am
here the first example of a 2 digits rank 0 logic found it eleven's sample.
In that specific case, eliminations don't touch the hardest step as seen by Sudoku Explainer
13.......7.6....54..4.9.......1.......5..4..348...9..7...3.16........87.5...8...9;ED=9.2/1.2/1.2
the solver stops the preliminary eliminations here
the 2 digits rank 0 found is the following ( "X" type)
and the reduced PM
In that specific case, eliminations don't touch the hardest step as seen by Sudoku Explainer
13.......7.6....54..4.9.......1.......5..4..348...9..7...3.16........87.5...8...9;ED=9.2/1.2/1.2
the solver stops the preliminary eliminations here
- Code: Select all
A B C |D E F |G H I
1 3 289 |24567 24567 2567 |279 2689 268
7 29 6 |28 1 238 |239 5 4
28 5 4 |267 9 2367 |1237 12368 1268
---------------------------------------------------
2369 2679 2379 |1 23567 25678 |2459 24689 268
269 12679 5 |2678 267 4 |129 12689 3
4 8 123 |256 2356 9 |125 126 7
---------------------------------------------------
289 2479 2789 |3 2457 1 |6 24 25
236 1246 123 |9 2456 256 |8 7 125
5 12467 127 |2467 8 267 |1234 1234 9
the 2 digits rank 0 found is the following ( "X" type)
- Code: Select all
8 Truths = {4R14 4C47 5R14 5C47 }
8 Links = {4r9 4b26 5b256 1n4 4n7 }
8 elims 2r1c4 6r1c4 7r1c4 2r4c7 9r4c7 5r6c5 4r9c2 4r9c8
and the reduced PM
- Code: Select all
floors PM 45
X X X |45+ 45+ 5+ |X X X <<
X X X |X X X |X X X
X X X |X X X |X X X
X X X |X 5+ 5+ |45+ 4+ X <<
X X X |X X X |X X X
X X X |5+ 5+ X |5+ X X
X 4+ X |X 45+ X |X 4+ 5+
X 4+ X |X 45+ 5+ |X X 5+
X 4+ X |4+ X X |4+ 4+ X
AA AA
- champagne
- 2017 Supporter
- Posts: 7465
- Joined: 02 August 2007
- Location: France Brittany
Re: exotic patterns below the potential hardest
by ronk » Mon Jun 24, 2013 8:10 am
champagne wrote:here the first example of a 2 digits rank 0 logic found it eleven's sample.
the 2 digits rank 0 found is the following ( "X" type)
....
- Code: Select all
8 Truths = {4R14 4C47 5R14 5C47 }
8 Links = {4r9 4b26 5b256 1n4 4n7 }
8 elims 2r1c4 6r1c4 7r1c4 2r4c7 9r4c7 5r6c5 4r9c2 4r9c8
Half of those truths are superfluous.
- Code: Select all
4 Truths = {5R4 4C47 5C4}
4 Links = {4r9 1n4 4n7 5b5}
8 Eliminations --> r1c4<>267, r9c28<>4, r4c7<>29, r6c5<>5
- ronk
- 2012 Supporter
- Posts: 4764
- Joined: 02 November 2005
- Location: Southeastern USA
Re: exotic patterns below the potential hardest
by champagne » Mon Jun 24, 2013 8:26 am
ronk wrote:Half of those truths are superfluous.
- Code: Select all
4 Truths = {5R4 4C47 5C4}
4 Links = {4r9 1n4 4n7 5b5}
8 Eliminations --> r1c4<>267, r9c28<>4, r4c7<>29, r6c5<>5
well done,
it could be the same in many of the rank 0 I find. When I select rows or columns as base, I always put as sets all the "floor" digits.
I'll store on the web the entire file of the rank 0 logic I found in eleven's sample (may be limited to the puzzles list to limit the size )
- champagne
- 2017 Supporter
- Posts: 7465
- Joined: 02 August 2007
- Location: France Brittany
Re: exotic patterns below the potential hardest
by ronk » Mon Jun 24, 2013 9:31 am
champagne wrote:it could be the same in many of the rank 0 I find. When I select rows or columns as base, I always put as sets all the "floor" digits.
I'll store on the web the entire file of the rank 0 logic I found in eleven's sample (may be limited to the puzzles list to limit the size )
I recommend you "limit the size" by filtering out those puzzles with simplistic loops. Loops without even one almost-naked-pair or one almost-hidden-pair will quickly turn off even an ardent sudoku-ist.
- ronk
- 2012 Supporter
- Posts: 4764
- Joined: 02 November 2005
- Location: Southeastern USA
Re: exotic patterns below the potential hardest
by champagne » Mon Jun 24, 2013 10:10 am
ronk wrote:champagne wrote:it could be the same in many of the rank 0 I find. When I select rows or columns as base, I always put as sets all the "floor" digits.
I'll store on the web the entire file of the rank 0 logic I found in eleven's sample (may be limited to the puzzles list to limit the size )
I recommend you "limit the size" by filtering out those puzzles with simplistic loops. Loops without even one almost-naked-pair or one almost-hidden-pair will quickly turn off even an ardent sudoku-ist.
IMO each potential user should rework that file using it's own criteria.
I thinks it's already a significant assistance to have cleared say 98% of the original file.
- champagne
- 2017 Supporter
- Posts: 7465
- Joined: 02 August 2007
- Location: France Brittany
Re: exotic patterns below the potential hardest
by David P Bird » Mon Jun 24, 2013 3:55 pm
There is a straightforward conjugate loop covering these eliminations:
(5)r4c7 = (5)r6c7 - (5)r6c4 = (5-4)r1c4 = (4)r9c4 - (4)r9c7 = (4)r4c7 – Loop
As conjugate loops are rank 0, I've tried to express them as MSLSs before but failed, so today after flailing around a bit I tried to express them as a hidden set and to my surprise succeeded .
MSHS Truths 1236R6, 12367R9, 45C47 (13 locked digits)
Links 1n4, 4n7, 6n34578, 9n234678 (13 cells)
=> r1c4 <> 267, r4c7 <> 29, r6c5 <> 5, r9c28 <> 5
This looks promising for me and anyone else that is trying to spot these openings by eye, but not necessarily for a computer solver. If this generalises then with some practice it would be possible to almost follow the conjugate chain this way which would save time when working out the eliminations that result. In following the chain strong links are expressed using the digits involved and weak links are expressed as strong links for the complementary set of digits (something I failed to get over to Blue earlier).
(5)r4c7 = (5)r6c7 - (5)r6c4 = (5-4)r1c4 = (4)r9c4 - (4)r9c7 = (4)r4c7 – Loop
As conjugate loops are rank 0, I've tried to express them as MSLSs before but failed, so today after flailing around a bit I tried to express them as a hidden set and to my surprise succeeded .
- Code: Select all
*----------------------------*----------------------------*----------------------------*
| <1> <3> 289 | (45)-267 24567 2567 | 279 2689 268 |
| <7> 29 <6> | 28 1 238 | 239 <5> <4> |
| 28 5 <4> | 267 <9> 2367 | 1237 12368 1268 |
*----------------------------*----------------------------*----------------------------*
| 2369 2679 2379 | <1> 23567 25678 | (45)-29 24689 268 |
| 269 12679 <5> | 2678 267 <4> | 129 12689 <3> |
1236 | <4> <8> (123) | (256) (236)-5 <9> | (125) (126) <7> |
*----------------------------*----------------------------*----------------------------*
| 289 2479 2789 | <3> 2457 <1> | <6> 24 25 |
| 236 1246 123 | 9 2456 256 | <8> <7> 125 |
12367 | <5> (1267)-4 (127) | (2467) <8> (267) | (1234) (123)-4 <9> |
*----------------------------*----------------------------*----------------------------*
45 45
MSHS Truths 1236R6, 12367R9, 45C47 (13 locked digits)
Links 1n4, 4n7, 6n34578, 9n234678 (13 cells)
=> r1c4 <> 267, r4c7 <> 29, r6c5 <> 5, r9c28 <> 5
This looks promising for me and anyone else that is trying to spot these openings by eye, but not necessarily for a computer solver. If this generalises then with some practice it would be possible to almost follow the conjugate chain this way which would save time when working out the eliminations that result. In following the chain strong links are expressed using the digits involved and weak links are expressed as strong links for the complementary set of digits (something I failed to get over to Blue earlier).
- David P Bird
- 2010 Supporter
- Posts: 1043
- Joined: 16 September 2008
- Location: Middle England
Re: exotic patterns below the potential hardest
by ronk » Wed Jun 26, 2013 11:29 pm
David P Bird wrote:There is a straightforward conjugate loop covering these eliminations:
(5)r4c7 = (5)r6c7 - (5)r6c4 = (5-4)r1c4 = (4)r9c4 - (4)r9c7 = (4)r4c7 – Loop
As conjugate loops are rank 0, I've tried to express them as MSLSs before but failed, so today after flailing around a bit I tried to express them as a hidden set and to my surprise succeeded .
...
MSHS Truths 1236R6, 12367R9, 45C47 (13 locked digits)
Links 1n4, 4n7, 6n34578, 9n234678 (13 cells)
=> r1c4 <> 267, r4c7 <> 29, r6c5 <> 5, r9c28 <> 5
Taking the ALS complements to the AHS within the conjugate loop produces this ...
- Code: Select all
11 Truths = {23569N4 123569N7}
11 Links = {4r9 5r6 2678c4 1239c7 7b3}
8 Eliminations --> r1c4<>267, r9c28<>4, r4c7<>29, r6c5<>5
... but the simplicity of the AHS loop can't be beaten.
- Code: Select all
4 Truths = {4C47 5C47}
4 Links = {4r9 5r6 1n4 4n7}
8 Eliminations --> r1c4<>267, r9c28<>4, r4c7<>29, r6c5<>5
- ronk
- 2012 Supporter
- Posts: 4764
- Joined: 02 November 2005
- Location: Southeastern USA
Re: exotic patterns below the potential hardest
by blue » Thu Jun 27, 2013 2:54 am
Hi David,
This is a not a conjugate loop, but a continuous loop.
(Eliminations from a conjugate loop are rank 1 -- like eliminations from discontinuous loops).
I understood what you were getting at, but I didn't like the way you were saying it.
You aren't replacing a weak link with some number of strong links, you're replacing it with K+1 (weak) cell links, and K "digit in house" truths (strong links), for some K.
This is an "Obi-Wahn transformation" thing: weak links for 'n' digits in a house with N empty cells and N free digits, can be replaced by N (weak) cell links and (N-n) (strong) "digit in house" truths. If the house contains hidden sets that don't use any of the 'n' digits, the value of 'N' can be reduced by the size of the hidden sets. In either case, the difference ... N - (N - n) ... (weak link count) - (strong link count) ... is always the same as the number of weak links being replaced. Another way of thinking about that, is that you're replacing 'n' (weak) "digit in house" links, by K+n (weak) cell links, with K=(N-n) digits locked in the cells, and the locked digits represented by the K strong links.
Best Regards,
Blue.
David P Bird wrote:There is a straightforward conjugate loop covering these eliminations:
(5)r4c7 = (5)r6c7 - (5)r6c4 = (5-4)r1c4 = (4)r9c4 - (4)r9c7 = (4)r4c7 – Loop
As conjugate loops are rank 0, (...)
This is a not a conjugate loop, but a continuous loop.
(Eliminations from a conjugate loop are rank 1 -- like eliminations from discontinuous loops).
In following the chain strong links are expressed using the digits involved and weak links are expressed as strong links for the complementary set of digits (something I failed to get over to Blue earlier).
I understood what you were getting at, but I didn't like the way you were saying it.
You aren't replacing a weak link with some number of strong links, you're replacing it with K+1 (weak) cell links, and K "digit in house" truths (strong links), for some K.
This is an "Obi-Wahn transformation" thing: weak links for 'n' digits in a house with N empty cells and N free digits, can be replaced by N (weak) cell links and (N-n) (strong) "digit in house" truths. If the house contains hidden sets that don't use any of the 'n' digits, the value of 'N' can be reduced by the size of the hidden sets. In either case, the difference ... N - (N - n) ... (weak link count) - (strong link count) ... is always the same as the number of weak links being replaced. Another way of thinking about that, is that you're replacing 'n' (weak) "digit in house" links, by K+n (weak) cell links, with K=(N-n) digits locked in the cells, and the locked digits represented by the K strong links.
Best Regards,
Blue.
- blue
- Posts: 1052
- Joined: 11 March 2013
27 posts
• Page 1 of 2 • 1, 2
