Finally had a response back from valve and multiplay.. the server configs are indeed correct. The connection problems have been caused by bugs in the game; so from a fixing the server perspective we're in the hands of valve.

In the meantime there is a workaround:

instead of creating a lobby and using 'sv_search_key sneaky' in the console, you can use:

ms_force_dedicated_server 85.236.100.37:27215

This forces the lobby to connect to the server directly and should allow randoms to join and fill slots (providing the lobby was set to public when the game was started)

A rather cumbersome command, but it's the best we can come up with for now.

* the bug is intermittant, so the other day after rebuilding the server I could connect several times in a row. The bug gets worse the busier steam is

Ok, so although there is a new way to connect to the server, how do we set up the configeration of the game settings, do we create a lobby set with the map and then connect using the command through the lobby?

Everything stays the same except instead of using sv_search_key sneaky you use ms_force_dedicated_server 85.236.100.37:27215 then click the start button

All the lobby creeation, settings and permissions are the same as usual.

Essentially all the command does is force the lobby to use the server specified instead of going looking for one.

ok, so when are we gonna play?