The New Sudoku Players' Forum

[enxio27]

The AMD processor is not a problem except for a limited number of new instructions.

All I know is that gridchecker refuses to run on my system. It exits with the following error:

Fatal Error: This program was not built to run on the processor in your system.
The allowed processors are: Intel(R) Pentium(R) 4 and compatible Intel processors with Streaming SIMD Extensions 3 (SSE3) instruction support.

Pentium 4?!?!? REALLY???

mladen updated the code with the last intel features and some versions of the AMD processors didn't have the adequate set.

This makes no sense to me. If a person is writing code for someone else to use, why not make it compatible with all major brands of processors? What is so special about the Intel instruction set that it warrants cutting out AMD users?

I had, about three years ago to change my own AMD processor the reach a full compatibility with the intel set of instructions used by mladen.

Did you have to perform your own compile to get it to work? I'm using an AMD A10-8700P processor with 10 cores. I can't imagine that's not new enough or powerful enough.

If you want to test your AMD compatibility, the easiest way would be to use my .exe. what can be done using mails or through a file loaded in a shared place.

Sure, I'll give it a try. Would Google Drive or Dropbox work for you?

[champagne]

If you want to test your AMD compatibility, the easiest way would be to use my .exe. what can be done using mails or through a file loaded in a shared place.

Sure, I'll give it a try. Would Google Drive or Dropbox work for you?

You can find in my google drive shared space here a file gridchecker_patgame.zip that should work for that. It has a readme file and an example of input output in the way I use it in the pattern game

The AMD processor is not a problem except for a limited number of new instructions.

All I know is that gridchecker refuses to run on my system. It exits with the following error:
Fatal Error: This program was not built to run on the processor in your system.
The allowed processors are: Intel(R) Pentium(R) 4 and compatible Intel processors with Streaming SIMD Extensions 3 (SSE3) instruction support.

mladen updated the code with the last intel features and some versions of the AMD processors didn't have the adequate set.

This makes no sense to me. If a person is writing code for someone else to use, why not make it compatible with all major brands of processors? What is so special about the Intel instruction set that it warrants cutting out AMD users?

I had, about three years ago to change my own AMD processor the reach a full compatibility with the intel set of instructions used by mladen.

Did you have to perform your own compile to get it to work? I'm using an AMD A10-8700P processor with 10 cores. I can't imagine that's not new enough or powerful enough.

First of all, Gridchecker AFAIK is the working platform of mladen. It is not a static program that you can easily share. If you have common interests, you can use all or part of it (and my 128 bit code is inherited from gridchecker ) .

Mladen tried long ago to improve the performances using the 128 bits set of instructions. AMD compatibility was very far of it's target and I have been the first victim of that. It seems, looking at your error message that he added a control of the processor used to avoid a bad use of the code.

And yes, in all uses of gridchecker that I made, I compiled the code in my microsoft visual c++

[enxio27]

You can find in my google drive shared space here a file gridchecker_patgame.zip that should work for that. It has a readme file and an example of input output in the way I use it in the pattern game

Thanks! I'll try it out.

UPDATE: It gave me a file of puzzles renumbered from the puzzles in the input file, but I don't see any ratings anywhere. How do I get those? (BTW, skfr has the same no-AMD issue as Mladen's compilation of gridchecker.)

[champagne]

You can find in my google drive shared space here a file gridchecker_patgame.zip that should work for that. It has a readme file and an example of input output in the way I use it in the pattern game

Thanks! I'll try it out.

UPDATE: It gave me a file of puzzles renumbered from the puzzles in the input file, but I don't see any ratings anywhere. How do I get those? (BTW, skfr has the same no-AMD issue as Mladen's compilation of gridchecker.)

Hi enxio27,

Having no idea of what you are looking for, it is difficult to answer in the proper way (from your view point) to your questions.

a) skfr is an old attempt to rate puzzles not too far from the pattern game referee (Sudoku Explainer) but much faster. We worked hard on that topic with mladen dobrichev, but this was years ago, with no compatibility issue between AMD and Intel.

b) here you have no rating just because the target was not to rate the puzzles, but to clear redundant puzzles.

When I play (played, I have no free core to-day for a long while), the sequence is the following

1) produce puzzles (usually vicinity batch)
2) kill redundant puzzles and put them in the data base of seen puzzles
3) Pre rate all puzzles
4) select puzzles to rate using skfr
5) select puzzles to submit to the referee (Sudoku Explainer)

Just to say that the rating comes later and that to go fast, you can have to do it step by step (BTW, patrice has a different strategy, using a modified version of Sudoku Explainer and having smaller batches)

If your target is the pattern game, each of these steps could be seen in details. What I sent you is the first phase of the step 2)

[enxio27]

Having no idea of what you are looking for, it is difficult to answer in the proper way (from your view point) to your questions.
When I play (played, I have no free core to-day for a long while), the sequence is the following

1) produce puzzles (usually vicinity batch)
2) kill redundant puzzles and put them in the data base of seen puzzles
3) Pre rate all puzzles
4) select puzzles to rate using skfr
5) select puzzles to submit to the referee (Sudoku Explainer)

So far, I have generated puzzles with the appropriate pattern using gsf and then rated them with Sudoku Explainer. As you and others have found, SE takes a long time, so I was hoping to either limit gsf's puzzle generation to (probable) pearls and diamonds (haven't figured out the correct command line options for that) or use something (gridchecker or skfr?) to pre-rate the puzzles to filter out all but (potential) pearls and diamonds. Then, my plan is to rate only the likely pearls/diamonds.

I'm just trying to make the process more efficient and more fruitful.

[dobrichev]

All I know is that gridchecker refuses to run on my system. It exits with the following error:

Fatal Error: This program was not built to run on the processor in your system.
The allowed processors are: Intel(R) Pentium(R) 4 and compatible Intel processors with Streaming SIMD Extensions 3 (SSE3) instruction support.

Pentium 4?!?!? REALLY???

Hi Enxio27,

I appreciate your efforts to join the Patterns game.

The error message clearly says that you need processor with SSE3 support.
I see that your processor clearly supports these instructions.

This message doesn't come from my code.
It is either injected by the compiler (yes, I used Intel compiler because it generated significantly better code than the MS compiler alternative from the same epoch), or comes from the OS.

What I know is that the code really uses SSE3 instructions for better performance.
If the error is fake (due to marketing reasons), then you should find a way to compile the code using another compiler.
Unfortunately for pure technical reasons I am not supporting Windows executables at the moment.

The latest source code, especially modified for playing Patterns Game, is available at https://github.com/dobrichev/gridchecker/tree/pg.
It has built-in skfr rating engine and does most of the technical work related to the game.
The tools which I personally used in the game are
- The gridchecker version mentioned above.
- The original command-line serate (link from the first page of the patterns game topic).
- A set of freely available tools for sorting and filtering the files like head, tail, sort, comm, cut.
- A 4-core home PC.

All this, plus few weeks of practice, plus some luck, are sufficient to win a game.

If you choose to use gridchecker for the Patterns Game and compile it for your platform, ask here and I will give you instructions which commands to use.

I can assist you how to make the code compatible to your platform after you reach a point where only few errors are still blocking the compilation.

Champagne answered the rest of your questions and I agree with him.

Cheers,
MD

[enxio27]

If you choose to use gridchecker for the Patterns Game and compile it for your platform, ask here and I will give you instructions which commands to use.

Mladen, thanks for your response. I now have champagne's compilation of gridchecker for AMD processor, and I have been able to get it to run. However, I'm not clear on how to get it to "pre-rate" and select puzzles for processing through SE (as required by PG). What command-line options do you recommend for this?

[dobrichev]

I now have champagne's compilation of gridchecker for AMD processor, and I have been able to get it to run.

Since I don't know what Champagne did with the code, it can or cannot work for the patterns game.
The easiest way is to try. If it doesn't work, you should compile (or ask Champagne to port to MS compiler and compile for you) the recent version.

What command-line options do you recommend for this?

Gridchecker is a command line tool. You should open one or more consoles and work there.
A brief list of the commands is listed when you run the tool without any command line parameters.
However, there are command combinations that still work but aren't listed properly on this screen.

So, I would recommend doing the following steps, with example from the recent pattern game 295.

1) create an empty directory and copy the tool there. Assume the executable is named gridchecker.exe. I am naming the directories to match the game number.
2) create a startup file with one or more puzzles for this particular game. A valid startup file would be the initial puzzle, in our example the 2.0/1.2/1.2 puzzle by Pat, copied from Patterns game results page.

echo "090080070008100003000000000400002000050090001003500020602300010010040002007005000" > 0

3) run gridchecker with the following command line

gridchecker --verbose --pattern --pg 3,0,120,0,120,0,120,3,1 < 0 > 1

This does:
- use file named "0" as input
- write results to file named "1"
- apply {-3,+3} to the filtered input puzzles (the first 3 after --pg)
- filter puzzles that have ER between 0.0 and 12.0 (the first pair 0,120) AND EP between 0..12 (the second pair 0,120) AND ED between 0..12 (the third pair 0,120)
- do maximum of 3 passes (the 3 after the filter pairs)
- ignore puzzles staring with single (last number, 0=don't ignore, 1=ignore)

The progress is displayed on the screen, something like this

Code: Select all

gridchecker --verbose --pattern --pg 3,0,120,0,120,0,120,3,1 < 0 > 1
Automorphism level 2
1 puzzles loaded
Relabel 3, 3 passes left, processing 1 items
Relabel 3, 2 passes left, processing 5 items
Relabel 3, 1 passes left, processing 160 items
6856 puzzles done

Total time 664.106 seconds.


4) investigate the generated puzzles. While the files are small, you can use textual editor (and prefer file names suffixed with .txt)
In this example 6855 new puzzles are generated + the original 1 = 6856.

This is how the result file 1 looks like

Hidden Text: Show

Code: Select all

.1..2..3...21....4.........1....5....3..6...2..74...8.4.17...9..8..9...6..5..3... ED=8.5/2.3/2.3 0 0 ED=0.0/0.0/0.0
.1..2..3...21....4.........1....5....6..3...7..47...8.9.12...4..4..7...3..5..6... ED=7.3/7.2/2.6 0 0 ED=0.0/0.0/0.0
....<other puzzles in the same format>...
.1..2..3...45....6.........7....8....9..1...5..64...2.8.96...1..5..4...9..3..7... ED=8.5/8.5/3.4 0 0 ED=0.0/0.0/0.0
.1..2..3...45....6.........7....8....9..1...5..69...4.8.96...1..5..7...2..7..9... ED=7.1/7.1/3.4 0 0 ED=0.0/0.0/0.0


Useful commands, not related to gridchecker, are

4.1) List the count of puzzles per ED

cat 1 | cut -f2 -d" "|cut -f2 -d= |cut -f3 -d"/" | sort -n|uniq -c

output

Hidden Text: Show

Code: Select all

    102 1.2
      2 1.5
    535 2.0
   1615 2.3
    259 2.5
   1927 2.6
    126 2.8
    374 3.0
     23 3.2
   1562 3.4
     10 3.6
     10 4.2
     10 4.4
     67 6.6
     24 6.7
      1 7.0
     87 7.1
     46 7.2
      4 7.3
      2 7.4
      1 7.5
      2 7.6
     38 7.8
      2 8.2
     11 8.3
      6 8.4
      1 8.5
      1 8.8
      4 8.9
      4 9.0


4.2) List the count of puzzles per ER

cat 1 | cut -f2 -d" "|cut -f2 -d= |cut -f1 -d"/" | sort -n|uniq -c

output

Hidden Text: Show

Code: Select all

     88 2.0
     30 2.3
     31 2.5
     17 2.6
     14 2.8
    120 3.0
      6 3.2
    165 3.4
     20 3.6
      1 3.8
     87 4.2
     19 4.4
     26 4.5
      2 4.6
      1 5.0
      8 5.6
    235 6.6
    205 6.7
     12 6.8
     13 6.9
     28 7.0
    897 7.1
   1310 7.2
    695 7.3
     42 7.4
     25 7.5
     77 7.6
    122 7.7
     85 7.8
      3 7.9
      1 8.0
     57 8.2
    732 8.3
    708 8.4
    286 8.5
     36 8.6
     37 8.7
    176 8.8
    233 8.9
    182 9.0
     18 9.1
      5 9.2
      1 9.3


So, after this step you have 4 puzzles rated x/y/9.0.
Not shown in the example, but they are 3 x 9.0/9.0/9.0 and 1 x 9.1/9.1/9.0

5) make decisions how to continue - define a filter and continue generation. A next step could be addition of ED filter >= 9.0.
gridchecker --verbose --pattern --pg 3,0,120,0,120,90,120,3,1 < 1 > 2
After seeing few more x/y/9.0 appeared and the generation was exhausted before the requested 3 passes, you may decide to relax the filter and continue with ED >= 8.8
gridchecker --verbose --pattern --pg 3,0,120,0,120,88,120,3,1 < 2 > 3
and see that on third generation in less than 10 minutes you have in hands this puzzle

Code: Select all

.1..2..3...43....5.........6....7....4..5...8..59...4.7.95...1..8..1...2..1..6... ED=9.5/9.5/9.5 3 0 ED=0.0/0.0/0.0


6) while waiting for the generation to finish, manually extract and re-rate the selected for submission puzzles using serate.

This looks sufficient for a first lesson.
Let see whether your port of gridchecker supports this command and we can continue latter.

Cheers,
MD

[enxio27]

This looks sufficient for a first lesson.
Let see whether your port of gridchecker supports this command and we can continue latter.

Thanks so much for this explanation! I will try it out with champagne's compilation and let you know how it goes. I had hoped that the next Patterns Game would have started by now, so that I could generate some puzzles for the game that I could also test with gridchecker, but it hasn't happened yet.

[dobrichev]

Today somehow this puzzle with no solutions appeared in my list,
and I don't know how it happened.

1..2....3.4..3..5...6...2..2....1.7..8..7.5.....4.......3.8.....5.7...649......2.

By chance the puzzles with no solution also can't be rated and this saved me.

once upon a time, i ran your software without specifying --minimals
and it did produce puzzles with 0 answers

i have now posted a nice example
-- may help in diagnosing

Pat's example is fixed in version v1.31 (2018-01-06)

[champagne]

Since I don't know what Champagne did with the code, it can or cannot work for the patterns game.

champagne did nothing else than to compile the code with MS visual basic, but this is an old version compiled in April 2016

[dobrichev]

Then it should work for the Patterns Game. Thank you.

[dobrichev]

I placed executables for 64-bit Windows and Linux on https://github.com/dobrichev/gridchecker/tree/master/binaries.

No stability nor full functional tests are done for Windows platform.

For Linux due to technical reasons (independence of OS version-specific shared libraries) I was forced to compile with changed compiler options and this resulted in ~25% SLOWER code. For those who want performance, compile from source with -flto option.

The Patterns Game branch wasn't updated.

[denis_berthier]

I downloaded gridchecker (from GitHub) and tried to compile it on my Mac (with "make").
(I deleted the -fopenmp option in the makefile, as MacOS doesn't know it.)

I get pages of warnings plus the following errors:

Code: Select all

fClueIterator.cpp:981:21: error: no member named 'emplace' in 'std::__1::queue<incomplete_puzzle_t,
      std::__1::deque<incomplete_puzzle_t, std::__1::allocator<incomplete_puzzle_t> > >'
        passedIncompleteBM.emplace(setClues, dc, clueNumber);

In file included from ./ch81.h:11:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/map:910:5: error:
      static_assert failed due to requirement 'is_same<std::__1::pair<bm128, int>, std::__1::pair<const
      bm128, int> >::value' "Allocator::value_type must be same type as value_type"
    static_assert((is_same<typename allocator_type::value_type, value_type>::value),

patterns.cpp:196:20: error: no type named 'const_iterator' in 'bm128IntMap'
                for(bm128IntMap::const_iterator i = ss.begin(); i != ss.end(); i++) {

subcanon.cpp:1:10: fatal error: 'malloc.h' file not found
#include <malloc.h>

templates.cpp:1842:30: error: a space is required between consecutive right angle brackets (use '> >')
        std::vector<std::vector<bool>> isFEP(46656, std::vector<bool>(46656,false));


I tried to change g++ to gcc, but it doesn't change anything.


[Edit]: Thanks to Mathimagics for providing a solution to my immediate need of minlexing patterns.
However, I'm still interested in being able to run gridchcker for other purposes, if the above errors on MacOS are not too complicated to overcome.

[dobrichev]

Hi Denis,

I will look at errors when I found time.

At first sight your compiler looks sufficiently new to know that malloc.h is considered non-standard and should be replaced by stdlib.h, and is sufficiently old not to know that emplace is improved version of push.
It is my duty to fix the code of course. The makefile could be outdated too.

You can help me by telling what of the versions of gridchecker you are interested in - the "standard" one from the master branch in github, or the one in pg branch, which has Patterns game functionalities.

Also, if you run
g++ -v
on your machine/environment, what version it reports (usually at the very top or at the end)?

A full list of errors - here in a hidden tag or in private message - are welcome too.

Cheers,
Mladen