The New Sudoku Players' Forum

[tarek]

I hope I made the correct call with those bug fixes.

The good news about the inconsistencies that you’ve mentioned that batch solving mode (in the next release) would solve it. It will require more independent testing obviously but all the initial tests shows no change in rating ER/EP/ED from non batch solving, consistent rating for those special cases and faster rating in general.

Batch solving will be helpful in rating large groups of puzzles for sorting reasons as well as for the patterns game. It will not eliminate the inconsistencies that you mentioned in the absence of that batch solving mode.

lksudoku has attempted addressing some of these but one of their bug fixes was inconsistent and wasn’t integrated for public use.

Tarek

[tarek]

I ran some tests on the 4000+ puzzle file from the patterns game posted by 1to9only that showed clearly how batch solving can give you a lower rating if there are several uniqueness hints available.

This is most likely the fault of the Uniqueness detection algorithms which fail if a specific element is removed (on this occasion an elimination by another uniqueness technique). When this happens; the solver goes for a more difficult technique and the puzzle gets a higher than required rating. Batch solving applies all the hints simultaneously and therefore the correct rating is given.

Code: Select all

.....4..3....2..79..7...25....6......7...5..19...1..26..3...8...52..6...18..43...,237,085,5.9/5.0/3.0,Patrice

This receives the same rating in the most recent release of Sukaku explainer. When you apply batch solving, however, the rating becomes:

Code: Select all

5.0/5.0/3.0   NQ/NQ/NP

[dobrichev]

... and therefore the correct rating is given.

I am not sure whether a rating system that gives smaller final rating on the cost of deeper/wider search is better than one that assumes analysing less steps ahead. Obviously the sequence of applied techniques matters, and this is part of the magic in the sudoku game.

[champagne]

I ran some tests on the 4000+ puzzle file from the patterns game posted by 1to9only that showed clearly how batch solving can give you a lower rating if there are several uniqueness hints available.

This is most likely the fault of the Uniqueness detection algorithms which fail if a specific element is removed (on this occasion an elimination by another uniqueness technique). When this happens; the solver goes for a more difficult technique and the puzzle gets a higher than required rating. Batch solving applies all the hints simultaneously and therefore the correct rating is given.

Code: Select all

.....4..3....2..79..7...25....6......7...5..19...1..26..3...8...52..6...18..43...,237,085,5.9/5.0/3.0,Patrice

This receives the same rating in the most recent release of Sukaku explainer. When you apply batch solving, however, the rating becomes:

Code: Select all

5.0/5.0/3.0   NQ/NQ/NP

SKFR and all my clones apply all hints simultaneously, this is part of the deviations with Sudoku Explainer (the main one in the low ratings). As far as I can remember, the final result is not always a rating below SE rating.

[tarek]

SKFR and all my clones apply all hints simultaneously, this is part of the deviations with Sudoku Explainer (the main one in the low ratings). As far as I can remember, the final result is not always a rating below SE rating.

Yes most of the ratings are identical (down to ER/EP/ED).
The few that are not identical (49 out 4000+ for this sample) show a reduction in in rating with batch solving.

In my opinion this shouldn't happen and I'm suspecting that it is due to the original coding of uniqueness type elimination. There are some rare occasions where puzzle isomorphs had a lower rating and other occasions inconsistencies in chain sorting resulted in selecting the longer chain … These also would be eliminated with batch solving and would result in a reduction of rating with batch solving.

I'll investigate further to see at what level did batch solving make a difference by simply disabling certain techniques (which should be possible with Sukaku explainer using the "--techs=TECHSTRING" option that will appear in future releases. I'm looking at identical ratings for both "non batch" solving and "batch solving" to determine that.

[tarek]

After running some tests. Unfortunately I've detected an issue with the original SE Chain detection highlighted by the difference in rating between batch solving and regular solving.

Look at this puzzle from the patterns game which was supplied in 1to9only's latest file

Code: Select all

.5...1...3...4.5.........9.....7...8.9.8.6.1.2...5.7...3...4.....4.6...7...7...8.;178;037;7.8/2.8/2.6;gsf

The current Sukaku explainer non-batch solving mode agrees with the rating. with batch solving mode 1 on ("-B1" option), the rating is 7.7/2.8.2.6

Here are the detailed eliminations which can be followed visually when you paste the before status into sudoku/sukaku explainer and press the "Next Hint" button repeatedly:

We reach this point in solving:

Code: Select all

.....6.8.....5.....2...6.8..23..6...........91...........4.6.8.......7....34.6.....3............7..........9.2...6......4............8.....5.....2...6...1........1....6.8....4.....12...6.8.....5.....23............7....3..6.8.........9.23..6.......5....1....6...1.3..6...1..4...........7...23.....9.2...6..9.234.6..........8....4.............9......7.........8..23...........6....23......1............5.....2.......1....6.8.1.3..6.8.1..4.........5......3.....9......7....34.6.....3..6..9......7....3......1...56....2......9.......8....4.....12...6....2..56....2...6..91......8912.....8....4......23.....9.....6....23.5....123.....9.23.5..........7.......6..9.2...6.......56.........7..1.........23.5.....234.6..9.......8..234.6..9
+-------------------+-------------------+-------------------+
| 68    5     268   | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  235   7     |
| 69    26    56    | 7     1     235   | 23469 8     23469 |
+-------------------+-------------------+-------------------+

Batch solving applies all the detected 3 hints that are rated 7.7 which leads to solving the puzzle with singles afterwards.
non-batch solving applies only the 1st of these 3 hints and you would expect that if the puzzle required a return to same point again that hint 2 and if needed hint 3 would be applied but this doesn't happen. The critical 7.7 hint is missed completely and a 7.8 rated hint is applied instead.

Batch solving: Show

Code: Select all

+-------------------+-------------------+-------------------+
| 68    5     268   | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  35    7     |
| 69    26    56    | 7     1     235   | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.7
%s: Nishio Forcing Chain: r8c8.2 on ==> r2c4.2 both on & off
+-------------------+-------------------+-------------------+
| 68    5     268   | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     2     7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     29    | 269   2346  8     |
| 4     9     7     | 8     3     6     | 2     1     5     |
| 2     168   1368  | 14    5     9     | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  35    7     |
| 69    26    56    | 7     1     235   | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.7
%s: Nishio Forcing Chain: r5c5.3 off ==> r8c4.3 both on & off
+-------------------+-------------------+-------------------+
| 68    5     268   | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     2     7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     29    | 269   2346  8     |
| 4     9     7     | 8     3     6     | 2     1     5     |
| 2     168   1368  | 14    5     9     | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  35    7     |
| 69    26    56    | 7     1     235   | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.7
%s: Nishio Forcing Chain: r5c7.3 on ==> r1c4.3 both on & off

nonBatch solving: Show

Code: Select all

+-------------------+-------------------+-------------------+
| 68    5     268   | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  35    7     |
| 69    26    56    | 7     1     235   | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.7
%s: Nishio Forcing Chain: r8c8.2 on ==> r2c4.2 both on & off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  35    7     |
| 69    26    56    | 7     1     235   | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.3
%s: Forcing Chain: r1c3.6 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 1239  35    7     |
| 69    26    56    | 7     1     35    | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.3
%s: Forcing Chain: r9c6.2 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 23469 8     23469 |
+-------------------+-------------------+-------------------+
%r: 6.6
%s: Turbot Fish: r8c7.2 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 126   256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 3469  8     23469 |
+-------------------+-------------------+-------------------+
%r: 6.7
%s: Forcing X-Chain: r9c7.2 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   2346  8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 16    256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 3469  8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.3
%s: Forcing Chain: r7c7.2 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     239   | 269   346   8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 16    256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 3469  8     23469 |
+-------------------+-------------------+-------------------+
%r: 2.8
%s: Claiming: Cells r4c7,r5c7: 2 in column and block
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    136   | 14    7     29    | 29    346   8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 16    256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 3469  8     23469 |
+-------------------+-------------------+-------------------+
%r: 3.4
%s: Hidden Pair: Cells r4c6,r4c7: 2,9 in row
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    13    | 14    7     29    | 29    346   8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 16    256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 3469  8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.2
%s: Forcing Chain: r4c3.6 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 236   9     1     | 468   7     346   |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     23    7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    13    | 14    7     29    | 29    346   8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 16    256   269   |
| 189   128   4     | 239   6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 346   8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.3
%s: Forcing Chain: r9c7.9 off
+-------------------+-------------------+-------------------+
| 68    5     28    | 3     9     1     | 468   7     46    |
| 3     7     9     | 26    4     8     | 5     26    1     |
| 168   4     1268  | 5     2     7     | 368   9     236   |
+-------------------+-------------------+-------------------+
| 5     16    13    | 14    7     29    | 29    346   8     |
| 4     9     7     | 8     23    6     | 23    1     5     |
| 2     168   1368  | 14    5     39    | 7     346   369   |
+-------------------+-------------------+-------------------+
| 7     3     156   | 29    8     4     | 16    256   269   |
| 189   128   4     | 29    6     235   | 139   35    7     |
| 69    26    56    | 7     1     35    | 346   8     23469 |
+-------------------+-------------------+-------------------+
%r: 7.8
%s: Nishio Forcing Chain: r1c4.3 off ==> r9c7.3 both on & off

This difference is not due to pattern destruction as the elements of that critical 7.7 rated hint are still present. So for me that is a bug and the puzzle should have been rated as 7.7 originally

[1to9only]

I've withdrawn the 4281 puzzles file, as in use I've noticed at least one error, and I think there are maybe 4-6 bad puzzles. Apologies if you've downloaded (and used) the file.

Code: Select all

...3..2.....7.8.1.....59..424.....5...1...7...6.....898..51.....3.4.7.....6..3... # 0154 206    4.9/4.2/2.6 ED=4.6/4.2/2.6


I've created a new list, it contains 4279 puzzles, and I'm going to revalidate them all with SE (will take some time!) before releasing an updated file.
.

[tarek]

I've withdrawn the 4281 puzzles file, as in use I've noticed at least one error, and I think there are maybe 4-6 bad puzzles. Apologies if you've downloaded (and used) the file.

Code: Select all

...3..2.....7.8.1.....59..424.....5...1...7...6.....898..51.....3.4.7.....6..3... # 0154 206    4.9/4.2/2.6 ED=4.6/4.2/2.6


I've created a new list, it contains 4279 puzzles, and I'm going to revalidate them all with SE (will take some time!) before releasing an updated file.
.

Ah thanks for letting me know

tarek

[champagne]

So for me that is a bug and the puzzle should have been rated as 7.7 originally

Having intensively tested my clones in the games, I had thousands examples of this bug
Part of the game was to find a puzzle rated 7.7 by SE. Not easy. This is why the entry 7.7 is challenging in the games

[tarek]

Thanks champagne,

If this has been seen before with skfr then we have to assume that ratings in their current form with ER 1.2.1.x are buggy and in cases will over rate puzzles. This may not have that dramatic effect on the Patterns Game but will have an impact on some puzzle collections with SE ratings used to sort puzzles.

How close are we to get a consistent and accurate rating.?

lksudoku's fixes with UR/UL loops; UR/UL type 3 sets and BUGs, Chain sorting (although short of perfect) in addition to batch solving could get us close to that. That may need to be the standard to sort the hardest from this moment on.

independent testing is very important and that is how we get bug reports or inconsistent cases reported.

A note about the next release of sukaku explainer:

The bug fixes for UR/UL loops detection; UR/UL type 3 sets detection and BUGs detection are integrated in the code now and can't be removed without re-coding. I made that decision after extensive testing that demonstrated these fixes to be consistent and always resulting in same or less ratings. We shouldn't accept inaccurate rating and if implementing these fixes changes our standard then our current standard needs to change IMO. The standards have been changed before by SE but they pre-date the Patterns game. I can't see a problem in turning the page and starting with new standards.

The Fix for chains length detection and sorting is inconsistent and therefore has not been hard coded into the project but is available as a hidden option (i.e. not supported) as it gave good results in most but not all cases. I will mention this option here but will not be available on the wiki help pages or the html --man help code yet.

Their will be an optional new rating as proposed elsewhere

Their will be optional control of solving techniques

I would hope that as many of the players here would download the next release and help with rating / testing. It is a useful tool as GUI too; with so many new features that would help for example: advanced players with hints after pasting the pencilmark state into the interface

[EDIT: Clarified a point about Sukaku explainer's integration of Uniqueness technique fixes]

[1to9only]

For patterns game addicts fans, the latest SukakuExplainer (1.4.0) release is OK to use at the risk of the occasional misrating due to lksudoku's BUG2 and UR/UL fixes. I'd advise making a custom version without lksudoku's fixes, all game 333 ratings are then as per SE.

The 1.4.0 SukakuExplainer (i.e. my own rebuild from the gitub source) works fine on puzzles from games 7 (all xx/1.0/1.0), 360 (xx/2.6/2.6 max) and 320 (some high ratings, but probably no BUG2 or UR/UL submissions that would cause a misrating).

I've (quick) tested mostly using these 4 patterns game puzzles.
.

[tarek]

For patterns game addicts fans

v1.4.0 is available to dowload https://github.com/SudokuMonster/SukakuExplainer/releases

Command line help page https://github.com/SudokuMonster/SukakuExplainer/wiki/Batch-mode-command-line-parameters

The new parameters

Code: Select all

  -N, --revisedRating=N
      Revised rating scheme. Default 0=disabled. 1=enabled
  -B, --batch=N
      Batch solving. Default 0=disabled.
                             1=Apply all available hints that have the lowest rating
                             2=Apply all available hints with rating lower than or equal to maximum puzzle rating
  -~, --techs=TECHSTRING
      Specific techniques only, set which techniques to use
      the techniques string TECHSTRING is a string consisting of the letters '0' and '1',
      where '1' means the technique should be used and '0' means it should not be used
      To see which technique is in which letter, and how many techniques are
      there, just type --techs= without any string after the = in TECHSTRING
  -T, --totalTime
      The time required to process all puzzles parsed to standard output
  -S, --showArguments
      Show parameters used

lksudoku's chain sorting/detection fix can be activated using the -C1 option which is not found on the Help page --man deocument.

A typical choice of parameters for me under windows with Batch solving Mode 2 would be

Code: Select all

-S -t0 -B2 -T --format="%%g;%%r/%%p/%%d;%%S/%%T/%%U;%%e"

If you decide to solve without using the -B option (non batch solving) similar to the standard use in GUI or the Patterns game then remember that the only difference would be the fix that handles the Uniqueness techniques bug(s) that have been extensively discussed. This may change any previously rated puzzle that had that bug at the time of rating.

-B1 & -B2 have resulted in faster rating & more consistent ratings. They should solve the isomorphic puzzle rating discrepancy that happens occasionally.

-B2 in initial testing was faster than -B1

GUI now has an increased ability of accepting pencilmark grids. You can now paste successfully into GUI most Pencilmark grids with markings and eliminations where it generated an error previously.

tarek

[Pat]

the great work by lksudoku
was done back in 2011


really great and wonderful,
and the Patterns Game could've adopted that version


but somehow,
stayed with the old Sudoku Explainer,
including various oddities (which may be regarded as bugs)


possibly the players prefer retaining these oddities

[Pat]

or if players wish to improve the rating,
i suspect they'd switch the game to champagne's SKFR

[tarek]

I hope that our current work recognises in part some of lksudoku's work which like all of what we do here is out of love of the game and out of our own personal time!

I think that the one that needs convincing is g.r.emlin! If g.r.emlin is convinced then things will start moving. Many of the improvements & work to improve / speed rating is based on the version of SE that used to referee the Patterns Game. Change the refereeing rules and players would adopt the new standard.

tarek