The New Sudoku Players' Forum

[1to9only]

I probably have a little bit of spare processing capacity in between patern games! Might download and try later!

Out of curiousity, and after a very long run, are we not just generating the same random grids? (a bit like in the patterns game!). I imagine that some (many) batches will overlap, but the goal is to scan the whole puzzle space (again a bit like the patterns game!).

[1to9only]

Can I obtain the source code for Gen19C.exe? To help debug 2 Worker deaths so far, each after some 9 hours of running!

Code: Select all

18:17:55: Generating initial seeds
...
19:06:53: Processing new items - pass #10
21:45:36: Items in = 717721, items out = 5896101
21:45:41: Sort/merge new items
21:45:42: Pass complete: ED grids/puzzles =   3411795
21:45:42: Processing new items - pass #11
03:22:56: Items in = 1556406, items out = 12936647
03:23:10: Sort/merge new items


Edit: [Windows exception] image deleted.

[1to9only]

I'm also looking at a custom compression tool that would give smaller zip files.

7zip produces smaller files than zip.

Batch001002.txt 654,407,275 (639MB)
Batch001002.zip 141,311,440 (138MB) <- using Zip 3.0 (July 5th 2008) from http://infozip.sourceforge.net/
Batch001002.7z 103,461,289 (101MB) <- using 7-Zip 19.00 (2019-02-21) from http://7-zip.org/
.

[1to9only]

I've had 3 Worker crashes in the 1st 24 hrs! I'm running the latest Gen19C.exe now. I had a quick look at the source code, could not spot anything wrong in ConsolidateList() where I think the crash occurred, likely fixed already. I'll run for another 24 hrs, and hopefully no more mishaps...

[1to9only]

This time ALL 4 Workers crashed. Processing is stopped. I'll do some investigations when time permits.

Worker 1: The instruction at 0x0000000000403154 referenced memory at 0x00000001A6012000. The memory could not be written.
Worker 2: The instruction at 0x0000000000403154 referenced memory at 0x00000001A601D000. The memory could not be written.
Worker 3: The instruction at 0x00000000004034A8 referenced memory at 0x00000001A6016000. The memory could not be written.
Worker 4: The instruction at 0x0000000000403154 referenced memory at 0x00000001A601F000. The memory could not be written.

Worker 3 stopped at a different code address.

I've had similar (occasional) failures (The memory could not be read.) when one of my programs usually ran 99.99% percent of the time, on this particular machine, and on other machines as well!
.

[1to9only]

I have 4 new Workers running. The next few hours should produce Batches!!

However, I've been able to reproduce the crash on BOTH yesterday's old code and today's new code, using the following values:

Code: Select all

#define MAXN     26056
#define TABSIZE 0x6000


The crash occurs within a few minutes of starting. [This is on a different machine from yesterday's.]

MAXN=12936648 and MAXN=26056 both allocate memory that leaves 80 unused bytes at the end of the allocated block. ConsolidateList() compares 81 bytes (less bytes on mismatches so may not get out of bounds) and then copies 166 bytes. This could lead to a crash if the MAXN+1 item is processed. I'm not even sure if this is a likely scenario!

My way forward will be to dump the allocated tables to files before calling ConsolidateList().

If a crash occurs, I'll have a modified program to load the tables from files before calling ConsolidateList(), and debugging this.
.

[1to9only]

I will discuss batch delivery options in a separate post.

Is this sorted? I have about 6.3GB of uncompressed batch data from a week ago. PC presently on patterns game duty. I don't want to keep filling up the disk space if I've no way of offloading some batches.

[1to9only]

The last time I had the Workers toiling away, I ended up hitting Ctrl-C to stop them and reclaim the Computer for other uses. The Batch data was lost. I'm going to modify my Workers to check for the STOPB signal after each pass and exit gracefully. No Batch data will be lost, but the Batches will be smaller.

Code: Select all

   while (1) {
      Pass++;
      CheckPause();
      LogEventN("Processing new items - pass #", Pass);
      ...
      gf = fopen("STOPB", "r");
      if (gf) { fclose(gf); break; }
   }


[1to9only]

In the previous Worker19C.c when this was provided, I make a couple of (private) changes:

This to speed up (I think!) batch making:

Code: Select all

      if ((NMT+np) >= MAXC) {NoMore = 1; np = MAXC-1; return;}  // failsafe


This for less frequent screen update, and a rough indication to batch completion:

Code: Select all

      if ((rn & 0x03ff) == 0) {printf("  at item %10d (%d%%)\r", rn, (NMT+np)/129366); fflush(stdout);}


[1to9only]

[deleted] - no longer relevant.

[1to9only]

the old (but soon to be "ex") PC

Can this PC be setup with an FTP server to allow results to be 'pushed' to you, rather than you currently 'pulling' from various locations (with Edge issues!)?

I've used FileZilla Server before, and it can be setup to only allow encrypted and named user logins with passwords, with read/write access to only to the user's folder. Uploaders would be using FZ client, it allows transfers to be paused and restarted.

In my own FZ setup, I still got occasional login attempts from white (or black) hats scanning for open ports, they'll try anonymous login or user/password and then disconnect!

[blue]

New info, possibly : Grids containing a 21 but no 20 ... awaiting 2nd party verification.

[blue]

Estimates suggest 30% of ED grids have 18C puzzles - so perhaps 2 billion

It's closer to 17.6%, or roughly one in 6, with ~60% of them having only one 18 -- see the end of this post.

Im thinking that not many grids will only have a solitary 20 .... most grids will have plenty of 20s ?

Average counts per grid are ~3.2 million (minimal) 20s (and ~2200 minimal 19s) -- see Afmob's results here.

--

I did some testing with random grids.
10,000,000 samples produced 461 grids with a 20 and no 19, putting the total count in the 229-276K range.
100,000,000 samples, and every one had a 20, putting the "21, but no 20" count at "no more than 164" (with 95% confidence, I think).

Edit: For "20, but no 19" it was 10M "non-automorphic" grids sampled, with 461 hits.
There are 4815 "automorphic" grids with a 20, but no 19 ... putting the total count in the 234-280K range.

[blue]

Hi Colin,

I did some testing with random grids.
10,000,000 samples produced 461 grids with a 20 and no 19.....

....
Is this an upper bound ? - or have you actually checked [with a "checking program"] that those 461 grids didnt have a 19 ? [i get the feeling you have]

Yes, and for good measure, I checked them to make sure they had a 20, too.

i'm thinking probability-wise that the number of grids without a 20 is a lot less than 164 ... maybe only 4 ?

It's certainly possible.
For "automorphic" grids the ratio of "no 19" to "no 20" grids was ~1200:1.
I didn't know what to expect, but I wondered if that might mean there were another ~4800 "non-automorphic" grids with no 20.
It's clear now, (for me, anyway), that there are nowhere near that many !

anyhow ive done a bit of research whilst mathemagic's program is ongoing ......

taking 8 random 18s - i generated similar pattern template 18s ... there appears to be a lower bound of an average of 26 puzzles per pattern
taking 8 random 19s - i generated similar pattern template 19s ... there appears to be a lower bound of an average of 10000 puzzles per pattern ... new ones still appearing though at the {+5} level

This makes the ratio between patterns with valid 18C puzzles / possible ED 18C patterns at smaller than 1/1000 which is difficult to understand if its correct !

The ratio is more like 1/95, using JPF's count of 184,060,159,680 ED patterns, and my estimate of 1,934,000,000 18C puzzles (mentioned earlier in the thread).
It seems like you've shown that if an 18C/19C pattern has one puzzle, then it's many times more likely that the same pattern has another puzzle, than it is that a randomly chosen pattern of the same size, has at least one puzzle.

ive been looking at a grid "which has an 18" but i've been struggling to find the 18 puzzle from scratch ....
this grid

Code: Select all

347981256582476193169523874896245317754318962213769485925137648478692531631854729

how long does your program take to confirm one 18-puzzle ? [ and how many 19s ? !]

5.4 seconds to confirm "only one" 18, and 213 seconds to find 2709 19's (including the 63 non-minimals).
(Any confirmation(s) of the 2709 number ?)

Cheers,
Blue.

[blue]

Exe files for Mathimagics ...

(deleted)