diff --git a/CLAUDE.md b/CLAUDE.md index 375c6ff..8cd853a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -22,10 +22,11 @@ continuity. | `make test-selfcheck` | Proves the runner catches each kind of failure, using `test/fixtures/`. | | `make smoke` | Boots the headless server for 1,000 ticks with 24 bots, prints tick timing, and checks exit codes and logs. Part of `make check`. | | `tools/godot.sh -- --server --seed=N --bots=N --ticks=N` | Runs a headless dedicated server. See `src/server/server_config.gd` for every option. Logs are JSON lines carrying the seed. Exit codes: 0 clean, 1 script error, 2 bad command line. | +| `tools/godot.sh -- --server --listen --port=N --bots=1` | Runs a server people can join over UDP (default port 24650). It waits in the lobby for the first of them and fills each match's empty places with bots. | | `tools/godot.sh ...` | Runs the pinned engine. Headless by default. | | `tools/capture.sh ` | Renders the real client offscreen in a throwaway `tidewall-capture-*` Docker container (Xvfb, Mesa software rendering) and saves a PNG. Fails on a blank frame or a renderer fallback. | | `tools/godot.sh --script res://tools/island_map.gd -- --seed=N --out=res://captures/island.png` | Draws a top-down map of a seed's island, with a contour at each tide level. | -| `make bot-matches` | Plays 20 bot-only matches (several minutes, under `nice`) and checks every one has a winner, under a tenth of deaths are drownings, easy bots place worst and hard bots take the most eliminations. `BOT_MATCH_SEEDS=5 BOT_MATCH_FIRST=40` changes the seeds. | +| `make bot-matches` | Plays 20 bot-only matches (about ten minutes, under `nice`) and checks every one has a winner, under a tenth of deaths are drownings, and easy bots win less than an even share and take the fewest eliminations. `BOT_MATCH_SEEDS=5 BOT_MATCH_FIRST=40` changes the seeds; under 20 matches the difficulty checks are skipped. | | `make match` | Plays one whole offline match, lobby to results: a scripted player and 23 bots on a local server (about 25 s). Saves the command log to `build/match/match.tclog`, replays it and checks the replay ends on the same state hash. `MATCH_SEED=N` changes the seed; the default, 2, is one where the scripted player also builds. | | `make match-captures` | Renders eight stages of that match from its command log (`captures/match_.png`), about 3 minutes. Needs Docker. Look at them. | | `make terrain-capture` | Renders the terrain crack check and a skirtless control in the capture container. The control must crack and the real terrain must not. Needs Docker. | diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index bd5aa42..ded73d6 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -145,19 +145,56 @@ and dropped. ## Snapshot format -A snapshot is a binary, little-endian packet: +`NetSnapshot` (`src/net/`) owns the layouts. An entity travels as a **record**: a list +of whole numbers, already quantized, laid out per archetype. The server compares +records to see what changed, the wire carries only the changed field groups, and the +client keeps records as its picture of the world. + +A snapshot is a binary, little-endian packet on channel 1, unreliable: | Part | Contents | |---|---| -| Header | Protocol version (u8), server tick (u32), baseline tick (u32, 0 = full), last command tick applied for this client (u32) | -| Tide | Phase (u8), level in centimetres (u16) | -| Entities | Count, then records of ID (varint), archetype (u8), changed-field mask (u8), then only the fields that changed | -| Removed | Count, then IDs | +| Header | Message type (u8), server tick (u32), baseline tick (u32, 0 = builds on nothing), simulation tick (u32), last command tick taken from this client (u32), the entity the client is watching (varint) | +| Match and tide | Match phase (u8), tide phase (u8), sea level in centimetres (s16) | +| Removed | Count (u8), then IDs (varint) | +| Entities | Count (u16), then records of ID (varint), archetype and changed-group mask (u8), then only the groups that changed | -Positions are quantized to 1.5 cm (u16 per axis, 1,024 m range), yaw to u16, health to u8. -Snapshots are deltas against the latest baseline the client acknowledged, and fall back to a full -snapshot if none. Players only receive entities in their relevance range. Building changes are also -sent on a reliable ordered event stream, since a dropped wall is a gameplay lie. +The server tick counts every tick the server has run and never goes back, so it +orders snapshots across matches; the simulation tick is the clock commands name. +Positions are quantized to 1/64 m (u16 per axis: 1,024 m across, heights from -128 m), +yaw to u16, pitch to s8, health and armour to u8. Field groups: players have position, +look, vitals, state flags and held item; pickups and shots have position and item. + +**Deltas and loss.** `SnapshotSender` remembers what it told each client in its last 40 +snapshots. Each snapshot is a delta against the newest one that client has +acknowledged (its input packets carry that tick), or builds on nothing if there is +none or it is too old. A delta only ever builds on something the client confirmed +holding, so a lost snapshot is never resent: the next one covers it. +`SnapshotReceiver` ignores snapshots older than its newest, and if it is ever sent a +delta on something it doesn't hold, acknowledges 0 to ask for a fresh start. + +**Interest.** A client is told only about entities near the player it is watching +(its own, or the one it spectates): players and shots within +`Tuning.NET_INTEREST_PLAYERS_METRES` (380 m, a little past the longest weapon) and +items on the ground within `NET_INTEREST_PICKUPS_METRES` (90 m), measured on the map. +An entity that leaves range is removed from the client's picture and nothing more is +said about it. Outside a match there is no viewer and a snapshot is only its header. + +**Budget.** A snapshot is at most `Tuning.NET_SNAPSHOT_MAX_BYTES` (1,200), so 20 a +second is at most 24 KB/s before packet headers and never fragments. When more has +changed than fits, the client's own player goes first, then the nearest, and each +snapshot an update has waited counts as 6 m closer, so nothing waits for ever. What +didn't fit is still owed and goes in a later snapshot. + +**Input.** The client answers on channel 1 with the snapshot tick it holds and its +last few commands (`Tuning.NET_INPUT_REDUNDANCY`), oldest first. `NetGame` +(`src/server/`) offers each tick's command to `ServerLoop.submit()` once. + +**Match notice.** On joining and at every change of phase the server sends, reliably +on channel 0, the match phase, index and seed, so the client can generate the island. + +Building changes will also travel on a reliable ordered event stream (N5), since a +dropped wall is a gameplay lie. ## Module map @@ -166,7 +203,7 @@ sent on a reliable ordered event stream, since a dropped wall is a gameplay lie. | `src/sim/` | Tick loop, entities, RNG, commands, tide, combat, building | `src/sim/` only | | `src/bots/` | Bot brains that produce `PlayerCommand`s | `src/sim/` | | `src/net/` | Transport, snapshot encoding, prediction, lag compensation | `src/sim/` | -| `src/server/` | Headless and listen-server hosting | `src/sim/`, `src/bots/`, `src/net/` | +| `src/server/` | Headless and listening servers, joining the network to the match | `src/sim/`, `src/bots/`, `src/net/` | | `src/client/` | World rendering, camera, input, effects | `src/sim/`, `src/net/` | | `src/ui/` | HUD, menus, map | `src/sim/` (read only), `src/client/` | | `src/audio/` | Sound playback, music | `src/sim/` (read only) | diff --git a/src/net/net_client.gd b/src/net/net_client.gd index aa82b41..26746de 100644 --- a/src/net/net_client.gd +++ b/src/net/net_client.gd @@ -33,6 +33,10 @@ var match_seed: int = 0 var max_players: int = 0 var tick_rate: int = 0 +## Packets from the server during the last poll(), once joined, in arrival order: +## {"channel", "bytes"}. Whoever owns the client decodes them. +var packets: Array[Dictionary] = [] + var _host: ENetConnection = null var _peer: ENetPacketPeer = null var _wanted_name: String = "" @@ -67,6 +71,7 @@ func connect_to(address: String, port: int, wanted_name: String, now_msec: int = func poll(now_msec: int = -1) -> void: + packets.clear() if _host == null: return var now: int = now_msec if now_msec >= 0 else Time.get_ticks_msec() @@ -128,7 +133,12 @@ func status_text() -> String: func _received(channel: int, bytes: PackedByteArray) -> void: - if channel != NetProtocol.CHANNEL_CONTROL or bytes.is_empty(): + if bytes.is_empty(): + return + if state == JOINED and (channel != NetProtocol.CHANNEL_CONTROL or bytes[0] != NetProtocol.MSG_REJECT): + packets.append({"channel": channel, "bytes": bytes}) + return + if channel != NetProtocol.CHANNEL_CONTROL: return if state == HANDSHAKING and bytes[0] == NetProtocol.MSG_WELCOME: var welcome: Dictionary = NetProtocol.decode_welcome(bytes) diff --git a/src/net/net_server.gd b/src/net/net_server.gd index 8ff81e8..9400851 100644 --- a/src/net/net_server.gd +++ b/src/net/net_server.gd @@ -20,6 +20,11 @@ var match_seed: int = 0 ## What happened during the last poll(): {"type": "joined", "handle", "name"}, ## {"type": "left", "handle", "name", "reason"} and {"type": "refused", "reason"}. var events: Array[Dictionary] = [] +## Game packets from joined players during the last poll(), in arrival order: +## {"handle", "bytes"}. Whoever owns the server decodes them. +var packets: Array[Dictionary] = [] +## The most game packets kept from one player in one poll(); the rest are dropped. +var max_packets_per_poll: int = 32 ## Malformed or unexpected packets dropped so far. var bad_packets: int = 0 @@ -33,6 +38,8 @@ var _players: Dictionary = {} ## ENetPacketPeer -> handle. var _handle_of: Dictionary = {} var _next_handle: int = 1 +## Handle -> game packets kept from them during this poll(). +var _packets_from: Dictionary = {} ## Starts listening on UDP `port` (0 picks a free one) on `address`. Returns OK or the @@ -82,6 +89,8 @@ func name_of(handle: int) -> String: ## Call it every frame. `now_msec` is for tests; it defaults to the engine's clock. func poll(now_msec: int = -1) -> void: events.clear() + packets.clear() + _packets_from.clear() if _host == null: return var now: int = now_msec if now_msec >= 0 else Time.get_ticks_msec() @@ -147,8 +156,15 @@ func _received(peer: ENetPacketPeer, channel: int, bytes: PackedByteArray) -> vo if _leaving.has(peer): return if _handle_of.has(peer): - # Game traffic arrives with N2. A second HELLO, or anything else, is dropped. - bad_packets += 1 + # Game traffic goes to the owner. A second HELLO, anything on the control + # channel, and a flood are dropped. + var handle: int = _handle_of[peer] + var count: int = int(_packets_from.get(handle, 0)) + if channel != NetProtocol.CHANNEL_STATE or bytes.is_empty() or count >= max_packets_per_poll: + bad_packets += 1 + return + _packets_from[handle] = count + 1 + packets.append({"handle": handle, "bytes": bytes}) return if not _guests.has(peer): return diff --git a/src/net/net_snapshot.gd b/src/net/net_snapshot.gd new file mode 100644 index 0000000..4b2d403 --- /dev/null +++ b/src/net/net_snapshot.gd @@ -0,0 +1,397 @@ +## Game traffic after the handshake: what a snapshot of the world looks like on the +## wire, the input packet a client answers with, and the match notice. +## +## An entity travels as a **record**: a PackedInt32Array of whole numbers, already +## quantized, laid out per archetype (see the R_ constants). The server compares records +## to decide what changed, the wire carries only the changed field groups, and the +## client keeps records as its picture of the world. Everything is little-endian. +## +## server -> SNAPSHOT (channel 1, unreliable) +## u8 type, u32 server tick, u32 baseline tick (0 = none), u32 simulation tick, +## u32 last command tick taken from this client, varint the entity the client is +## watching (its own player, or the one it spectates; 0 = none), u8 match +## phase, u8 tide phase, s16 sea level in centimetres, +## u8 removed count, then entity IDs as varints, +## u16 record count, then per record: varint ID, u8 (archetype << 5 | group mask), +## then the fields of each group in the mask +## client -> INPUT (channel 1, unreliable) +## u8 type, u32 tick of the newest snapshot it holds (0 = send a full one), +## u8 command count, then that many 14-byte PlayerCommands, oldest first +## server -> MATCH (channel 0, reliable) +## u8 type, u8 match phase, u32 match index, s64 match seed +## +## Decoders return an empty Dictionary for anything malformed and never raise an error. +class_name NetSnapshot +extends RefCounted + +const MSG_MATCH: int = 4 +const MSG_SNAPSHOT: int = 16 +const MSG_INPUT: int = 17 + +## The server tick counts every tick the server has run and never goes back, so it +## orders snapshots across matches. The simulation tick is the one commands name. +const HEADER_BYTES: int = 17 +const MATCH_BYTES: int = 14 +## The most commands one INPUT may carry. Clients send fewer; see Tuning. +const MAX_INPUT_COMMANDS: int = 8 + +## Positions are quantized to 1/64 m (about 1.5 cm) in a u16 per axis: 1,024 m across, +## and heights from Y_OFFSET metres below zero to 1,024 less that above. +const POSITION_STEPS_PER_METRE: float = 64.0 +const Y_OFFSET_METRES: float = 128.0 + +# Record layouts. Index 0 is always the archetype and 1 to 3 the position. +const R_ARCHETYPE: int = 0 +const R_X: int = 1 +const R_Y: int = 2 +const R_Z: int = 3 +# Players. +const R_YAW: int = 4 +## Pitch in steps of 1/127 of a right angle. +const R_PITCH: int = 5 +const R_HEALTH: int = 6 +const R_ARMOR: int = 7 +const R_FLAGS: int = 8 +## The held item's catalog code plus one; 0 is the boathook. +const R_HELD: int = 9 +# Pickups and projectiles: the item's catalog code plus one. +const R_ITEM: int = 4 +# Pickups. +const R_COUNT: int = 5 +const R_TIER: int = 6 +# Projectiles: the entity that fired it. +const R_OWNER: int = 5 + +const FLAG_ON_GROUND: int = 1 << 0 +const FLAG_CROUCHING: int = 1 << 1 +const FLAG_SWIMMING: int = 1 << 2 +const FLAG_ELIMINATED: int = 1 << 3 +## Two bits: Drop.NONE, ABOARD, FREEFALL or GLIDING. +const FLAG_DROP_SHIFT: int = 4 +const FLAG_RELOADING: int = 1 << 6 +const FLAG_USING: int = 1 << 7 + +const _U8: int = 0 +const _S8: int = 1 +const _U16: int = 2 +const _U32: int = 3 +const _WIDTH: Array[int] = [1, 1, 2, 4] + +## Per archetype: the wire type of every record index. +const _TYPES: Dictionary = { + SimEntity.ARCHETYPE_PLAYER: [_U8, _U16, _U16, _U16, _U16, _S8, _U8, _U8, _U8, _U8], + SimEntity.ARCHETYPE_PICKUP: [_U8, _U16, _U16, _U16, _U8, _U16, _U8], + SimEntity.ARCHETYPE_PROJECTILE: [_U8, _U16, _U16, _U16, _U8, _U32], +} +## Per archetype: the record indices in each field group. Bit n of a mask is group n. +const _GROUPS: Dictionary = { + SimEntity.ARCHETYPE_PLAYER: [[1, 2, 3], [4, 5], [6, 7], [8], [9]], + SimEntity.ARCHETYPE_PICKUP: [[1, 2, 3], [4, 5, 6]], + SimEntity.ARCHETYPE_PROJECTILE: [[1, 2, 3], [4, 5]], +} + + +static func known_archetype(archetype: int) -> bool: + return _TYPES.has(archetype) + + +## The mask with every group of `archetype` set. +static func full_mask(archetype: int) -> int: + return (1 << (_GROUPS[archetype] as Array).size()) - 1 + + +## The record for an entity as it stands, or an empty array for an archetype that isn't +## replicated. +static func record_of(entity: SimEntity) -> PackedInt32Array: + var record: PackedInt32Array = PackedInt32Array() + if not _TYPES.has(entity.archetype): + return record + record.resize((_TYPES[entity.archetype] as Array).size()) + record[R_ARCHETYPE] = entity.archetype + record[R_X] = _quantize(entity.position.x) + record[R_Y] = _quantize(entity.position.y + Y_OFFSET_METRES) + record[R_Z] = _quantize(entity.position.z) + match entity.archetype: + SimEntity.ARCHETYPE_PLAYER: + record[R_YAW] = posmod(entity.yaw, PlayerCommand.YAW_STEPS) + record[R_PITCH] = clampi(roundi(float(entity.pitch) / PlayerCommand.PITCH_MAX * 127.0), -127, 127) + record[R_HEALTH] = clampi(entity.health, 0, 255) + record[R_ARMOR] = clampi(entity.armor, 0, 255) + var flags: int = (entity.drop_state & 3) << FLAG_DROP_SHIFT + if entity.on_ground: + flags |= FLAG_ON_GROUND + if entity.crouching: + flags |= FLAG_CROUCHING + if entity.swimming: + flags |= FLAG_SWIMMING + if entity.eliminated: + flags |= FLAG_ELIMINATED + if entity.reload_ticks > 0: + flags |= FLAG_RELOADING + if entity.use_ticks > 0: + flags |= FLAG_USING + record[R_FLAGS] = flags + var held: int = -1 + if entity.selected >= 1 and entity.selected <= entity.slot_items.size(): + held = entity.slot_items[entity.selected - 1] + record[R_HELD] = clampi(held + 1, 0, 255) + SimEntity.ARCHETYPE_PICKUP: + record[R_ITEM] = clampi(entity.item + 1, 0, 255) + record[R_COUNT] = clampi(entity.item_count, 0, 65535) + record[R_TIER] = clampi(entity.item_tier, 0, 255) + SimEntity.ARCHETYPE_PROJECTILE: + record[R_ITEM] = clampi(entity.item + 1, 0, 255) + record[R_OWNER] = entity.owner_id & 0xFFFFFFFF + return record + + +## Where a record's entity is, to the nearest 1/64 m. +static func position_of(record: PackedInt32Array) -> Vector3: + return Vector3( + record[R_X] / POSITION_STEPS_PER_METRE, + record[R_Y] / POSITION_STEPS_PER_METRE - Y_OFFSET_METRES, + record[R_Z] / POSITION_STEPS_PER_METRE, + ) + + +static func yaw_radians_of(record: PackedInt32Array) -> float: + return float(record[R_YAW]) / PlayerCommand.YAW_STEPS * TAU + + +static func pitch_radians_of(record: PackedInt32Array) -> float: + return float(record[R_PITCH]) / 127.0 * (PI * 0.5) + + +## The groups in which two records of one archetype differ. +static func diff_mask(before: PackedInt32Array, after: PackedInt32Array) -> int: + var groups: Array = _GROUPS[after[R_ARCHETYPE]] + var mask: int = 0 + for group: int in groups.size(): + for index: int in groups[group]: + if before[index] != after[index]: + mask |= 1 << group + break + return mask + + +## Bytes one record takes on the wire with `mask`. +static func record_bytes(entity_id: int, archetype: int, mask: int) -> int: + var types: Array = _TYPES[archetype] + var groups: Array = _GROUPS[archetype] + var total: int = varint_bytes(entity_id) + 1 + for group: int in groups.size(): + if mask & (1 << group) != 0: + for index: int in groups[group]: + total += _WIDTH[types[index]] + return total + + +static func varint_bytes(value: int) -> int: + var total: int = 1 + while value >= 0x80: + value >>= 7 + total += 1 + return total + + +## Builds a SNAPSHOT. `removed` are entity IDs; `records` is an array of +## [entity ID, mask, record] in the order they should travel. +static func encode_snapshot(header: Dictionary, removed: PackedInt32Array, records: Array) -> PackedByteArray: + var out: StreamPeerBuffer = StreamPeerBuffer.new() + out.big_endian = false + out.put_u8(MSG_SNAPSHOT) + out.put_u32(int(header["tick"]) & 0xFFFFFFFF) + out.put_u32(int(header.get("baseline", 0)) & 0xFFFFFFFF) + out.put_u32(int(header.get("sim_tick", 0)) & 0xFFFFFFFF) + out.put_u32(int(header.get("command_tick", 0)) & 0xFFFFFFFF) + _put_varint(out, int(header.get("own_id", 0))) + out.put_u8(int(header.get("match_phase", 0))) + out.put_u8(int(header.get("tide_phase", 0))) + out.put_16(clampi(int(header.get("sea_cm", 0)), -32768, 32767)) + out.put_u8(removed.size()) + for entity_id: int in removed: + _put_varint(out, entity_id) + out.put_u16(records.size()) + for entry: Array in records: + var record: PackedInt32Array = entry[2] + var archetype: int = record[R_ARCHETYPE] + var mask: int = entry[1] + var types: Array = _TYPES[archetype] + var groups: Array = _GROUPS[archetype] + _put_varint(out, int(entry[0])) + out.put_u8(archetype << 5 | mask) + for group: int in groups.size(): + if mask & (1 << group) == 0: + continue + for index: int in groups[group]: + match types[index]: + _U8: + out.put_u8(record[index]) + _S8: + out.put_8(record[index]) + _U16: + out.put_u16(record[index]) + _U32: + out.put_u32(record[index]) + return out.data_array + + +## Reads a SNAPSHOT's header and lists, without applying them: {"tick", "baseline", +## "sim_tick", "command_tick", "own_id", "match_phase", "tide_phase", "sea_cm", "removed": +## PackedInt32Array, "records": [[entity ID, archetype, mask, values]]}, where `values` +## holds the fields present, in record order. Empty if malformed. +static func decode_snapshot(bytes: PackedByteArray) -> Dictionary: + if bytes.size() < HEADER_BYTES + 7 or bytes[0] != MSG_SNAPSHOT: + return {} + var cursor: Array[int] = [HEADER_BYTES] + var own_id: int = _read_varint(bytes, cursor) + if own_id < 0 or cursor[0] + 5 > bytes.size(): + return {} + var at: int = cursor[0] + var result: Dictionary = { + "tick": bytes.decode_u32(1), "baseline": bytes.decode_u32(5), "sim_tick": bytes.decode_u32(9), + "command_tick": bytes.decode_u32(13), + "own_id": own_id, "match_phase": bytes[at], "tide_phase": bytes[at + 1], "sea_cm": bytes.decode_s16(at + 2), + } + var removed_count: int = bytes[at + 4] + cursor[0] = at + 5 + var removed: PackedInt32Array = PackedInt32Array() + for _index: int in removed_count: + var entity_id: int = _read_varint(bytes, cursor) + if entity_id <= 0: + return {} + removed.append(entity_id) + if cursor[0] + 2 > bytes.size(): + return {} + var record_count: int = bytes.decode_u16(cursor[0]) + cursor[0] += 2 + var records: Array = [] + for _index: int in record_count: + var entity_id: int = _read_varint(bytes, cursor) + if entity_id <= 0 or cursor[0] >= bytes.size(): + return {} + var tag: int = bytes[cursor[0]] + cursor[0] += 1 + var archetype: int = tag >> 5 + var mask: int = tag & 0x1F + if not _TYPES.has(archetype) or mask == 0 or mask > full_mask(archetype): + return {} + var types: Array = _TYPES[archetype] + var groups: Array = _GROUPS[archetype] + var values: PackedInt32Array = PackedInt32Array() + for group: int in groups.size(): + if mask & (1 << group) == 0: + continue + for index: int in groups[group]: + var type: int = types[index] + var offset: int = cursor[0] + if offset + _WIDTH[type] > bytes.size(): + return {} + match type: + _U8: + values.append(bytes[offset]) + _S8: + values.append(bytes.decode_s8(offset)) + _U16: + values.append(bytes.decode_u16(offset)) + _U32: + values.append(bytes.decode_u32(offset)) + cursor[0] = offset + _WIDTH[type] + records.append([entity_id, archetype, mask, values]) + if cursor[0] != bytes.size(): + return {} + result["removed"] = removed + result["records"] = records + return result + + +## Lays decoded `values` over `base` (an existing record, or empty for a new entity, +## which needs every group). Returns the new record, or an empty array if it can't be. +static func merge_record(base: PackedInt32Array, archetype: int, mask: int, values: PackedInt32Array) -> PackedInt32Array: + var types: Array = _TYPES[archetype] + var groups: Array = _GROUPS[archetype] + var record: PackedInt32Array = base.duplicate() + if record.is_empty(): + if mask != full_mask(archetype): + return PackedInt32Array() + record.resize(types.size()) + record[R_ARCHETYPE] = archetype + elif record[R_ARCHETYPE] != archetype: + return PackedInt32Array() + var next: int = 0 + for group: int in groups.size(): + if mask & (1 << group) == 0: + continue + for index: int in groups[group]: + record[index] = values[next] + next += 1 + return record + + +static func encode_input(ack_tick: int, commands: Array[PlayerCommand]) -> PackedByteArray: + var count: int = mini(commands.size(), MAX_INPUT_COMMANDS) + var bytes: PackedByteArray = PackedByteArray() + bytes.resize(6) + bytes[0] = MSG_INPUT + bytes.encode_u32(1, ack_tick & 0xFFFFFFFF) + bytes[5] = count + for index: int in range(commands.size() - count, commands.size()): + bytes.append_array(commands[index].encode()) + return bytes + + +## {"ack", "commands": Array[PlayerCommand]} or empty. +static func decode_input(bytes: PackedByteArray) -> Dictionary: + if bytes.size() < 6 or bytes[0] != MSG_INPUT: + return {} + var count: int = bytes[5] + if count > MAX_INPUT_COMMANDS or bytes.size() != 6 + count * PlayerCommand.ENCODED_SIZE: + return {} + var commands: Array[PlayerCommand] = [] + for index: int in count: + commands.append(PlayerCommand.decode(bytes, 6 + index * PlayerCommand.ENCODED_SIZE)) + return {"ack": bytes.decode_u32(1), "commands": commands} + + +static func encode_match(phase: int, match_index: int, match_seed: int) -> PackedByteArray: + var bytes: PackedByteArray = PackedByteArray() + bytes.resize(MATCH_BYTES) + bytes[0] = MSG_MATCH + bytes[1] = phase + bytes.encode_u32(2, match_index & 0xFFFFFFFF) + bytes.encode_s64(6, match_seed) + return bytes + + +## {"phase", "index", "seed"} or empty. +static func decode_match(bytes: PackedByteArray) -> Dictionary: + if bytes.size() != MATCH_BYTES or bytes[0] != MSG_MATCH or bytes[1] >= Match.PHASE_NAMES.size(): + return {} + return {"phase": bytes[1], "index": bytes.decode_u32(2), "seed": bytes.decode_s64(6)} + + +static func _quantize(metres: float) -> int: + return clampi(roundi(metres * POSITION_STEPS_PER_METRE), 0, 65535) + + +static func _put_varint(out: StreamPeerBuffer, value: int) -> void: + value &= 0xFFFFFFFF + while value >= 0x80: + out.put_u8(value & 0x7F | 0x80) + value >>= 7 + out.put_u8(value) + + +## Reads a varint of at most five bytes at cursor[0] and moves the cursor past it. +## Returns -1 if the buffer ends first or the number is too long. +static func _read_varint(bytes: PackedByteArray, cursor: Array[int]) -> int: + var value: int = 0 + for shift: int in range(0, 35, 7): + if cursor[0] >= bytes.size(): + return -1 + var byte: int = bytes[cursor[0]] + cursor[0] += 1 + value |= (byte & 0x7F) << shift + if byte & 0x80 == 0: + return value & 0xFFFFFFFF + return -1 diff --git a/src/net/net_snapshot.gd.uid b/src/net/net_snapshot.gd.uid new file mode 100644 index 0000000..259ebdb --- /dev/null +++ b/src/net/net_snapshot.gd.uid @@ -0,0 +1 @@ +uid://bu03pks5esll4 diff --git a/src/net/snapshot_receiver.gd b/src/net/snapshot_receiver.gd new file mode 100644 index 0000000..7675e11 --- /dev/null +++ b/src/net/snapshot_receiver.gd @@ -0,0 +1,99 @@ +## The client's side of snapshots: rebuilds the part of the world the server chose to +## tell it about, and says which snapshot it holds so the server can send changes only. +## +## Snapshots may arrive late, twice, out of order or not at all. One older than the +## newest applied is ignored. One built on a snapshot this client no longer holds can't +## be used, so the next acknowledgement asks the server to start again. +class_name SnapshotReceiver +extends RefCounted + +## Snapshots kept to apply later ones against. More than the server keeps, so anything +## it builds on is still here. +const HISTORY: int = Tuning.NET_SNAPSHOT_HISTORY + 24 + +## The newest picture: entity ID -> record (see NetSnapshot for the layout). +var entities: Dictionary = {} +## The server tick of that picture, or 0 before the first snapshot. +var tick: int = 0 +## The simulation tick of that picture: the clock commands are stamped against. +var sim_tick: int = 0 +## This client's own entity, or the one it is spectating; 0 outside a match. +var own_id: int = 0 +## The newest of this client's command ticks the server had taken. +var command_tick: int = 0 +var match_phase: int = 0 +var tide_phase: int = 0 +var sea_level: float = 0.0 +## Entities that appeared or went in the last applied snapshot. +var added: PackedInt32Array = PackedInt32Array() +var removed: PackedInt32Array = PackedInt32Array() +## Counts, for tests and the network graph. +var applied: int = 0 +var ignored_old: int = 0 +var missing_baseline: int = 0 +var malformed: int = 0 + +var _history: Dictionary = {} +var _need_full: bool = false + + +## The tick to acknowledge in the next INPUT: the newest snapshot held, or 0 to ask +## for one that builds on nothing. +func ack_tick() -> int: + return 0 if _need_full else tick + + +## Applies one SNAPSHOT packet. Returns true if it became the newest picture. +func apply(bytes: PackedByteArray) -> bool: + var snapshot: Dictionary = NetSnapshot.decode_snapshot(bytes) + if snapshot.is_empty(): + malformed += 1 + return false + var new_tick: int = snapshot["tick"] + var baseline_tick: int = snapshot["baseline"] + if new_tick <= tick: + ignored_old += 1 + return false + if baseline_tick != 0 and not _history.has(baseline_tick): + missing_baseline += 1 + _need_full = true + return false + var state: Dictionary = (_history[baseline_tick] as Dictionary).duplicate() if baseline_tick != 0 else {} + var gone: PackedInt32Array = PackedInt32Array() + for entity_id: int in snapshot["removed"]: + state.erase(entity_id) + var fresh: PackedInt32Array = PackedInt32Array() + for entry: Array in snapshot["records"]: + var entity_id: int = entry[0] + var base: PackedInt32Array = state.get(entity_id, PackedInt32Array()) + var record: PackedInt32Array = NetSnapshot.merge_record(base, entry[1], entry[2], entry[3]) + if record.is_empty(): + malformed += 1 + _need_full = true + return false + state[entity_id] = record + for entity_id: int in state: + if not entities.has(entity_id): + fresh.append(entity_id) + for entity_id: int in entities: + if not state.has(entity_id): + gone.append(entity_id) + entities = state + added = fresh + removed = gone + tick = new_tick + sim_tick = snapshot["sim_tick"] + own_id = snapshot["own_id"] + command_tick = snapshot["command_tick"] + match_phase = snapshot["match_phase"] + tide_phase = snapshot["tide_phase"] + sea_level = int(snapshot["sea_cm"]) / 100.0 + _need_full = false + applied += 1 + _history[new_tick] = state + while _history.size() > HISTORY: + var oldest: int = new_tick + for kept: int in _history: + oldest = mini(oldest, kept) + _history.erase(oldest) + return true diff --git a/src/net/snapshot_receiver.gd.uid b/src/net/snapshot_receiver.gd.uid new file mode 100644 index 0000000..a12f8b6 --- /dev/null +++ b/src/net/snapshot_receiver.gd.uid @@ -0,0 +1 @@ +uid://uw03hw8iwugs diff --git a/src/net/snapshot_sender.gd b/src/net/snapshot_sender.gd new file mode 100644 index 0000000..245ba4a --- /dev/null +++ b/src/net/snapshot_sender.gd @@ -0,0 +1,193 @@ +## The server's side of snapshots: decides, for each client, which entities it may know +## about, what has changed since the last snapshot that client confirmed, and what fits +## in this one. +## +## Each client's snapshot is a delta against the newest snapshot it has acknowledged +## (its INPUT packets carry that tick). The server remembers what it told each client +## in the last Tuning.NET_SNAPSHOT_HISTORY snapshots; if the acknowledged one isn't +## among them, or the client asks, it starts again from nothing. Because a delta only +## ever builds on something the client confirmed holding, a lost snapshot needs no +## resend: the next one covers it. +## +## Interest: a client is told only about entities within range of the player it is +## watching, measured on the map (height is ignored). An entity that leaves range is +## removed from the client's picture, and nothing about it is sent until it returns. +class_name SnapshotSender +extends RefCounted + +## Client handle -> {"acked": tick, "history": {tick: {entity ID: record}}, +## "last_sent": {entity ID: tick}, "command_tick": int, "bytes": int, "snapshots": int, +## "full": int, "deferred": int}. +var _clients: Dictionary = {} +## Records made for the tick being sent, shared between clients: entity ID -> record. +var _records: Dictionary = {} +var _records_tick: int = -1 +## Every entity at that tick, listed once for all clients. +var _entities: Array[SimEntity] = [] + + +func add_client(handle: int) -> void: + _clients[handle] = { + "acked": 0, "history": {}, "last_sent": {}, "command_tick": 0, + "bytes": 0, "snapshots": 0, "full": 0, "deferred": 0, + } + + +func remove_client(handle: int) -> void: + _clients.erase(handle) + + +func has_client(handle: int) -> bool: + return _clients.has(handle) + + +## Notes the snapshot tick a client says it holds. 0 asks for a fresh start. +func acknowledge(handle: int, tick: int) -> void: + if not _clients.has(handle): + return + var client: Dictionary = _clients[handle] + if tick == 0 or tick > int(client["acked"]): + client["acked"] = tick + + +## Notes the newest command tick taken from a client, echoed in its snapshots. +func note_command(handle: int, command_tick: int) -> void: + if _clients.has(handle): + var client: Dictionary = _clients[handle] + client["command_tick"] = maxi(int(client["command_tick"]), command_tick) + + +## Forgets what every client was told: a new match has a new simulation, whose entity +## IDs start again. +func reset() -> void: + for handle: int in _clients: + var client: Dictionary = _clients[handle] + client["acked"] = 0 + client["history"] = {} + client["last_sent"] = {} + client["command_tick"] = 0 + _records.clear() + _entities.clear() + _records_tick = -1 + + +## {"bytes", "snapshots", "full", "deferred"} sent to a client so far. "deferred" +## counts entity updates that had to wait for a later snapshot. +func stats(handle: int) -> Dictionary: + if not _clients.has(handle): + return {} + var client: Dictionary = _clients[handle] + return {"bytes": client["bytes"], "snapshots": client["snapshots"], "full": client["full"], "deferred": client["deferred"]} + + +## The entity IDs a client watching `viewer` may be told about now: the viewer, and +## everything replicated within range of it. Empty when there's no viewer. +## `among` is the list to choose from, when the caller already has every entity. +static func interest(sim: Simulation, viewer: SimEntity, among: Array[SimEntity] = []) -> Array[int]: + var visible: Array[int] = [] + if viewer == null: + return visible + if among.is_empty(): + among = sim.entities.entities() + var near: float = Tuning.NET_INTEREST_PICKUPS_METRES * Tuning.NET_INTEREST_PICKUPS_METRES + var far: float = Tuning.NET_INTEREST_PLAYERS_METRES * Tuning.NET_INTEREST_PLAYERS_METRES + var centre: Vector3 = viewer.position + for entity: SimEntity in among: + if not NetSnapshot.known_archetype(entity.archetype): + continue + var dx: float = entity.position.x - centre.x + var dz: float = entity.position.z - centre.z + var limit: float = near if entity.archetype == SimEntity.ARCHETYPE_PICKUP else far + if entity.id == viewer.id or dx * dx + dz * dz <= limit: + visible.append(entity.id) + return visible + + +## Builds the snapshot for a client at the simulation's current tick. `viewer_id` is +## the entity the client is watching (its own player, or the one it spectates), or 0 +## outside a match. `tick` is the server's own tick count, which must grow with every +## call for the same client. +func build(handle: int, sim: Simulation, viewer_id: int, tick: int, match_phase: int) -> PackedByteArray: + if not _clients.has(handle): + return PackedByteArray() + var client: Dictionary = _clients[handle] + var history: Dictionary = client["history"] + var baseline_tick: int = int(client["acked"]) + if not history.has(baseline_tick): + baseline_tick = 0 + var base: Dictionary = history.get(baseline_tick, {}) + var viewer: SimEntity = sim.entities.find(viewer_id) if viewer_id > 0 else null + if _records_tick != tick: + _records.clear() + _records_tick = tick + _entities = sim.entities.entities() + + var header: Dictionary = { + "tick": tick, "baseline": baseline_tick, "sim_tick": sim.tick, "command_tick": client["command_tick"], + "own_id": viewer.id if viewer != null else 0, "match_phase": match_phase, + "tide_phase": sim.tide.phase_at(sim.tick) if sim.tide != null else 0, + "sea_cm": roundi(sim.sea_level * 100.0), + } + var visible: Dictionary = {} + # [score, entity ID, mask, record]: the lowest score travels first. + var changed: Array = [] + var last_sent: Dictionary = client["last_sent"] + var interval: int = Tuning.NET_SNAPSHOT_INTERVAL_TICKS + for entity_id: int in interest(sim, viewer, _entities): + var record: PackedInt32Array = _record(sim, entity_id) + visible[entity_id] = true + var mask: int = NetSnapshot.full_mask(record[NetSnapshot.R_ARCHETYPE]) + if base.has(entity_id): + mask = NetSnapshot.diff_mask(base[entity_id], record) + if mask == 0: + continue + var score: float = -1.0e9 + if entity_id != viewer.id: + var waited: float = float(tick - int(last_sent.get(entity_id, tick - interval * 20))) / interval + score = NetSnapshot.position_of(record).distance_to(viewer.position) - waited * Tuning.NET_STALE_METRES_PER_SNAPSHOT + changed.append([score, entity_id, mask, record]) + changed.sort_custom(func(a: Array, b: Array) -> bool: return a[0] < b[0] if a[0] != b[0] else a[1] < b[1]) + + var state: Dictionary = base.duplicate() + var budget: int = Tuning.NET_SNAPSHOT_MAX_BYTES - NetSnapshot.HEADER_BYTES - NetSnapshot.varint_bytes(int(header["own_id"])) - 7 + var removed: PackedInt32Array = PackedInt32Array() + for entity_id: int in base: + if visible.has(entity_id): + continue + var cost: int = NetSnapshot.varint_bytes(entity_id) + if removed.size() >= 255 or cost > budget: + break + removed.append(entity_id) + state.erase(entity_id) + last_sent.erase(entity_id) + budget -= cost + var records: Array = [] + for entry: Array in changed: + var record: PackedInt32Array = entry[3] + var cost: int = NetSnapshot.record_bytes(entry[1], record[NetSnapshot.R_ARCHETYPE], entry[2]) + if cost > budget: + client["deferred"] += 1 + continue + budget -= cost + records.append([entry[1], entry[2], record]) + state[entry[1]] = record + last_sent[entry[1]] = tick + + history[tick] = state + if history.size() > Tuning.NET_SNAPSHOT_HISTORY: + var oldest: int = tick + for kept: int in history: + oldest = mini(oldest, kept) + history.erase(oldest) + var bytes: PackedByteArray = NetSnapshot.encode_snapshot(header, removed, records) + client["bytes"] += bytes.size() + client["snapshots"] += 1 + if baseline_tick == 0: + client["full"] += 1 + return bytes + + +func _record(sim: Simulation, entity_id: int) -> PackedInt32Array: + if not _records.has(entity_id): + _records[entity_id] = NetSnapshot.record_of(sim.entities.find(entity_id)) + return _records[entity_id] diff --git a/src/net/snapshot_sender.gd.uid b/src/net/snapshot_sender.gd.uid new file mode 100644 index 0000000..ba37d66 --- /dev/null +++ b/src/net/snapshot_sender.gd.uid @@ -0,0 +1 @@ +uid://ngjmvhpqyy2g diff --git a/src/server/headless_server.gd b/src/server/headless_server.gd index 055ce76..fd21b23 100644 --- a/src/server/headless_server.gd +++ b/src/server/headless_server.gd @@ -17,6 +17,8 @@ const FAULT_TICK: int = 10 var config: ServerConfig var loop: ServerLoop +## The network side, when the server was started with --listen. +var game: NetGame = null var _errors: ErrorCounter = ErrorCounter.new() var _accumulator: float = 0.0 @@ -39,20 +41,38 @@ func _ready() -> void: if not config.has_seed: config.match_seed = _fresh_seed() Engine.max_fps = 0 if config.unthrottled else REALTIME_MAX_FPS - loop = ServerLoop.new(config.match_seed, config.players, config.bots, null, config.bot_difficulty) + var rules: MatchRules = null + if config.listen: + rules = MatchRules.new() + rules.fill_with_bots = config.bots > 0 + loop = ServerLoop.new(config.match_seed, config.players, config.bots, rules, config.bot_difficulty) + if config.listen: + var net: NetServer = NetServer.new() + var error: Error = net.listen(config.port) + if error != OK: + _log("listen_failed", {"port": config.port, "message": "couldn't listen on UDP port %d: %s" % [config.port, error_string(error)]}) + _finish(2) + return + game = NetGame.new(loop, net) var started: Dictionary = config.to_dictionary() started["engine"] = Engine.get_version_info()["string"] started["tick_rate_hz"] = Tuning.TICK_RATE_HZ started["world_hash"] = loop.sim.island.heightmap_hash() started["match_phase"] = loop.match_flow.phase_name() - started["listening"] = false - started["note"] = "networking arrives in v0.5; players join as idle slots until then" + started["listening"] = game != null _log("start", started) func _process(delta: float) -> void: if _finished or loop == null: return + if game != null: + game.poll() + for event: Dictionary in game.events: + var fields: Dictionary = {"handle": event["handle"], "name": event["name"], "players": game.net.player_count()} + if event["type"] == "left": + fields["reason"] = event["reason"] + _log("player_" + String(event["type"]), fields) var due: int = UNTHROTTLED_BATCH if not config.unthrottled: _accumulator += delta @@ -79,6 +99,8 @@ func _tick() -> void: if config.fault == "script-error" and loop.ticks == FAULT_TICK: _deliberate_script_error() loop.run_tick() + if game != null: + game.after_tick() for event: Dictionary in loop.match_flow.events: if event["type"] in ["phase", "match_started", "match_ended"]: _log("match", event) @@ -94,6 +116,8 @@ func _tick() -> void: func _finish(requested_code: int) -> void: _finished = true var code: int = requested_code + if game != null: + game.net.close() if loop != null: var stats: Dictionary = loop.timing_stats() stats["state_hash"] = loop.sim.state_hash() diff --git a/src/server/net_game.gd b/src/server/net_game.gd new file mode 100644 index 0000000..fa08bd5 --- /dev/null +++ b/src/server/net_game.gd @@ -0,0 +1,111 @@ +## Joins the transport to the match: people who connect become participants, their +## input packets become commands through ServerLoop.submit(), and everyone is sent +## snapshots of what they may see, 20 times a second. +## +## Call poll() once a frame before running ticks, and after_tick() after each tick. +class_name NetGame +extends RefCounted + +var loop: ServerLoop +var net: NetServer +var sender: SnapshotSender = SnapshotSender.new() +## What happened during the last poll(): {"type": "joined" or "left", "handle", "name", +## "reason"}, for the server's log. +var events: Array[Dictionary] = [] +## Input packets that couldn't be read, so far. +var bad_inputs: int = 0 + +## Handle -> the newest command tick taken from them this match. +var _last_command: Dictionary = {} + + +func _init(server_loop: ServerLoop, server_net: NetServer) -> void: + loop = server_loop + net = server_net + net.max_players = loop.match_flow.rules.max_players + net.match_seed = loop.match_flow.current_seed() + + +## Services the network: joins and leaves, then every input packet that arrived. +func poll() -> void: + events.clear() + net.poll() + # kick() adds to net.events, so walk a copy. + for event: Dictionary in net.events.duplicate(): + var handle: int = int(event.get("handle", 0)) + if event["type"] == "joined": + if loop.match_flow.join(handle): + sender.add_client(handle) + _last_command[handle] = -1 + _send_match(handle) + events.append(event) + else: + net.kick(handle, NetProtocol.REASON_FULL) + elif event["type"] == "left" and sender.has_client(handle): + loop.match_flow.leave(handle) + sender.remove_client(handle) + _last_command.erase(handle) + events.append(event) + for packet: Dictionary in net.packets: + var handle: int = packet["handle"] + if not sender.has_client(handle): + continue + var input: Dictionary = NetSnapshot.decode_input(packet["bytes"]) + if input.is_empty(): + bad_inputs += 1 + continue + sender.acknowledge(handle, int(input["ack"])) + if not loop.match_flow.in_match(): + continue + # Packets repeat recent commands; each tick's command is offered once, whether + # or not the simulation takes it. + for command: PlayerCommand in input["commands"]: + if command.tick <= int(_last_command[handle]): + continue + _last_command[handle] = command.tick + if loop.submit(handle, command): + sender.note_command(handle, command.tick) + + +## Tells everyone about a change of phase, and sends snapshots when one is due. +func after_tick() -> void: + var flow: Match = loop.match_flow + var changed: bool = false + for event: Dictionary in flow.events: + if event["type"] == "phase": + changed = true + if event["to"] == "lobby": + # A new simulation: entity IDs and ticks start again. + sender.reset() + for handle: int in _last_command: + _last_command[handle] = -1 + net.match_seed = flow.current_seed() + if changed: + for handle: int in net.handles(): + _send_match(handle) + if loop.ticks % Tuning.NET_SNAPSHOT_INTERVAL_TICKS != 0: + return + for handle: int in net.handles(): + if not sender.has_client(handle): + continue + var bytes: PackedByteArray = sender.build(handle, loop.sim, viewer_of(handle), loop.ticks, flow.phase) + net.send(handle, NetProtocol.CHANNEL_STATE, bytes, false) + + +## The entity a participant is watching: their own player, the player they spectate +## once eliminated, or 0 when they aren't in a match. +func viewer_of(handle: int) -> int: + var flow: Match = loop.match_flow + if flow.phase == Match.LOBBY or flow.phase == Match.COUNTDOWN or not flow.entity_of.has(handle): + return 0 + var target: int = flow.spectate_target(handle) + if target != 0 and flow.entity_of.has(target): + return int(flow.entity_of[target]) + return int(flow.entity_of[handle]) + + +func _send_match(handle: int) -> void: + var flow: Match = loop.match_flow + var started: bool = flow.in_match() or flow.phase == Match.RESULTS + var index: int = flow.matches_started - 1 if started else flow.matches_started + net.send(handle, NetProtocol.CHANNEL_CONTROL, NetSnapshot.encode_match(flow.phase, index, flow.current_seed()), true) diff --git a/src/server/net_game.gd.uid b/src/server/net_game.gd.uid new file mode 100644 index 0000000..bcb8770 --- /dev/null +++ b/src/server/net_game.gd.uid @@ -0,0 +1 @@ +uid://b3uihs71yhq6f diff --git a/src/server/server_config.gd b/src/server/server_config.gd index fe813a9..a6b678c 100644 --- a/src/server/server_config.gd +++ b/src/server/server_config.gd @@ -1,9 +1,13 @@ ## Command-line configuration for a headless server. ## -## tools/godot.sh -- --server [--seed=N] [--port=N] [--players=N] [--bots=N] +## tools/godot.sh -- --server [--listen] [--port=N] [--seed=N] [--players=N] [--bots=N] ## [--bot-difficulty=mixed|easy|normal|hard] [--ticks=N] ## [--unthrottled] [--log-every=N] ## +## With `--listen` the server takes network players on UDP `--port`, waits in the lobby +## for the first of them, and with any `--bots` above 0 fills each match's empty places +## with bots. Without it the match starts at once with `--players` idle slots. +## ## Unknown or malformed arguments are errors, so a typo can't silently run the wrong ## match. See `parse()`. class_name ServerConfig @@ -19,9 +23,11 @@ var server: bool = false ## The match seed. `has_seed` is false when none was given and the server must pick one. var match_seed: int = 0 var has_seed: bool = false -## UDP port the server will listen on once networking lands (v0.5). +## True when `--listen` was given: take network players on `port`. +var listen: bool = false +## UDP port a listening server uses. var port: int = DEFAULT_PORT -## Player slots for human clients. Until v0.5 they join as idle players. +## Idle player slots, for a server that isn't listening. var players: int = 0 var bots: int = 0 ## Every bot's difficulty, as an index into Tuning.BOT_DIFFICULTIES, or -1 to mix them. @@ -59,6 +65,7 @@ func is_valid() -> bool: func to_dictionary() -> Dictionary: return { "seed": match_seed, + "listen": listen, "port": port, "players": players, "bots": bots, @@ -72,6 +79,9 @@ func _parse_one(arg: String) -> void: if arg == "--unthrottled": unthrottled = true return + if arg == "--listen": + listen = true + return var parts: PackedStringArray = arg.split("=", true, 1) if parts.size() != 2 or not arg.begins_with("--"): errors.append("unknown argument '%s'" % arg) @@ -129,6 +139,8 @@ func _validate() -> void: "--players plus --bots is %d, more than the match size of %d" % [players + bots, Tuning.SOLO_PLAYERS] ) + if listen and players > 0: + errors.append("--players is for a server that isn't listening; with --listen, people join over the network") if ticks < 0: errors.append("--ticks can't be negative") if log_every < 0: diff --git a/src/sim/tuning.gd b/src/sim/tuning.gd index f635bae..0c791a4 100644 --- a/src/sim/tuning.gd +++ b/src/sim/tuning.gd @@ -10,6 +10,25 @@ const MAX_COMMAND_LEAD_TICKS: int = 60 ## Width and depth of the playable world. Positions are clamped inside it. const WORLD_SIZE_METRES: float = 1024.0 +# Network. See docs/ARCHITECTURE.md, "Snapshot format". +## Server ticks between snapshots: 20 a second. +const NET_SNAPSHOT_INTERVAL_TICKS: int = 3 +## The most bytes in one snapshot. Keeps a packet inside one UDP datagram on any +## ordinary link, and a client under 24 KB/s before packet headers. +const NET_SNAPSHOT_MAX_BYTES: int = 1200 +## A client is sent other players and shots within this many metres on the map of the +## player it is watching: a little further than the longest weapon reaches. +const NET_INTEREST_PLAYERS_METRES: float = 380.0 +## And items lying on the ground within this many. +const NET_INTEREST_PICKUPS_METRES: float = 90.0 +## Snapshots the server remembers per client, to send changes against. +const NET_SNAPSHOT_HISTORY: int = 40 +## When a snapshot is full, each snapshot an entity has waited counts as this many +## metres closer, so nothing far away waits for ever. +const NET_STALE_METRES_PER_SNAPSHOT: float = 6.0 +## Commands a client repeats in each input packet, so a lost packet costs nothing. +const NET_INPUT_REDUNDANCY: int = 3 + # Match sizes const SOLO_PLAYERS: int = 24 const DUO_TEAMS: int = 12 diff --git a/test/net/test_net_handshake.gd b/test/net/test_net_handshake.gd index 0ebb8ec..926a945 100644 --- a/test/net/test_net_handshake.gd +++ b/test/net/test_net_handshake.gd @@ -10,6 +10,8 @@ var clients: Array[NetClient] = [] ## Every server event since the test began. var seen: Array[Dictionary] = [] var raw_hosts: Array[ENetConnection] = [] +## Game packets the server handed to its owner since the test began. +var handed: int = 0 func before_each() -> void: @@ -17,6 +19,7 @@ func before_each() -> void: server.match_seed = 4242 clients.clear() seen.clear() + handed = 0 raw_hosts.clear() assert_eq(server.listen(0, HOST), OK) assert_true(server.port() > 0) @@ -43,6 +46,7 @@ func _pump(done: Callable, limit_msec: int = 4000) -> bool: while Time.get_ticks_msec() - started < limit_msec: server.poll() seen.append_array(server.events) + handed += server.packets.size() for client: NetClient in clients: client.poll() for host: ENetConnection in raw_hosts: @@ -242,11 +246,13 @@ func test_random_packets_from_guests_and_players_never_crash_the_server() -> voi for index: int in bytes.size(): bytes[index] = rng.randi_range(0, 255) peer.send(rng.randi_range(0, 1), bytes, ENetPacketPeer.FLAG_RELIABLE) - # A player who keeps sending HELLOs and junk stays a player and changes nothing. + # A player who keeps sending HELLOs and junk stays a player and changes nothing. The + # HELLOs are dropped; junk on the game channel goes to the owner, who can't read it. for _packet: int in 20: member._peer.send(NetProtocol.CHANNEL_CONTROL, NetProtocol.encode_hello("Someone Else"), ENetPacketPeer.FLAG_RELIABLE) member._peer.send(NetProtocol.CHANNEL_STATE, PackedByteArray([1, 2, 3]), ENetPacketPeer.FLAG_RELIABLE) - assert_true(_pump(func() -> bool: return _events("refused").size() >= 12 and server.bad_packets >= 52), "refused %d, dropped %d" % [_events("refused").size(), server.bad_packets]) + assert_true(_pump(func() -> bool: return _events("refused").size() >= 12 and server.bad_packets >= 32 and handed >= 20), "refused %d, dropped %d, handed on %d" % [_events("refused").size(), server.bad_packets, handed]) + assert_eq(handed, 20) assert_eq(server.player_count(), 1) assert_eq(server.name_of(1), "Mara") assert_eq(_events("joined").size(), 1) diff --git a/test/net/test_net_snapshot.gd b/test/net/test_net_snapshot.gd new file mode 100644 index 0000000..15f96a0 --- /dev/null +++ b/test/net/test_net_snapshot.gd @@ -0,0 +1,138 @@ +extends TestCase + +## Issue N2: the wire format of snapshots, input packets and the match notice. + + +func _player(sim: Simulation, at: Vector3) -> SimEntity: + var player: SimEntity = sim.entities.find(sim.add_player()) + player.position = at + return player + + +func test_a_record_keeps_position_to_a_sixty_fourth_of_a_metre() -> void: + var sim: Simulation = Simulation.new(3, SimGround.flat()) + var player: SimEntity = _player(sim, Vector3(511.37, -20.21, 1003.99)) + player.yaw = 40000 + player.pitch = -PlayerCommand.PITCH_MAX + player.health = 73 + player.crouching = true + var record: PackedInt32Array = NetSnapshot.record_of(player) + assert_true(NetSnapshot.position_of(record).distance_to(player.position) < 0.014) + assert_near(NetSnapshot.yaw_radians_of(record), float(40000) / 65536.0 * TAU, 0.0001) + assert_near(NetSnapshot.pitch_radians_of(record), -PI * 0.5, 0.0001) + assert_eq(record[NetSnapshot.R_HEALTH], 73) + assert_true(record[NetSnapshot.R_FLAGS] & NetSnapshot.FLAG_CROUCHING != 0) + assert_eq(record[NetSnapshot.R_HELD], 0, "the boathook") + + +func test_positions_outside_the_world_are_clamped_not_wrapped() -> void: + var sim: Simulation = Simulation.new(3, SimGround.flat()) + var player: SimEntity = _player(sim, Vector3(-50.0, 5000.0, 2000.0)) + var where: Vector3 = NetSnapshot.position_of(NetSnapshot.record_of(player)) + assert_near(where.x, 0.0, 0.001) + assert_near(where.z, 65535.0 / 64.0, 0.001) + assert_near(where.y, 65535.0 / 64.0 - NetSnapshot.Y_OFFSET_METRES, 0.001) + + +func test_a_snapshot_round_trips_every_archetype_and_partial_masks() -> void: + var sim: Simulation = Simulation.new(3, SimGround.flat()) + var player: SimEntity = _player(sim, Vector3(100.0, 0.0, 100.0)) + var pickup: SimEntity = sim.entities.find(Inventory.spawn_pickup(sim, Vector3(101.0, 0.0, 100.0), 1, 7, -1, 2)) + var shot: SimEntity = sim.entities.create(SimEntity.ARCHETYPE_PROJECTILE) + shot.position = Vector3(120.0, 9.0, 80.0) + shot.item = 4 + shot.owner_id = player.id + var before: PackedInt32Array = NetSnapshot.record_of(player) + player.position.x += 3.0 + player.health = 12 + var after: PackedInt32Array = NetSnapshot.record_of(player) + var mask: int = NetSnapshot.diff_mask(before, after) + assert_eq(mask, 0b101, "position and vitals changed, nothing else") + var header: Dictionary = { + "tick": 900, "baseline": 897, "sim_tick": 300, "command_tick": 298, "own_id": player.id, + "match_phase": Match.PLAY, "tide_phase": 2, "sea_cm": -150, + } + var records: Array = [ + [player.id, mask, after], + [pickup.id, NetSnapshot.full_mask(SimEntity.ARCHETYPE_PICKUP), NetSnapshot.record_of(pickup)], + [shot.id, NetSnapshot.full_mask(SimEntity.ARCHETYPE_PROJECTILE), NetSnapshot.record_of(shot)], + ] + var bytes: PackedByteArray = NetSnapshot.encode_snapshot(header, PackedInt32Array([70000, 5]), records) + var expected: int = NetSnapshot.HEADER_BYTES + 1 + 4 + 1 + 4 + 2 + for entry: Array in records: + expected += NetSnapshot.record_bytes(entry[0], entry[2][0], entry[1]) + assert_eq(bytes.size(), expected, "record_bytes() predicts the encoded size") + var back: Dictionary = NetSnapshot.decode_snapshot(bytes) + for key: String in header: + assert_eq(back[key], header[key], key) + assert_eq(back["removed"], PackedInt32Array([70000, 5])) + assert_eq((back["records"] as Array).size(), 3) + var merged: PackedInt32Array = NetSnapshot.merge_record(before, SimEntity.ARCHETYPE_PLAYER, back["records"][0][2], back["records"][0][3]) + assert_eq(merged, after) + assert_eq(NetSnapshot.merge_record(PackedInt32Array(), SimEntity.ARCHETYPE_PICKUP, back["records"][1][2], back["records"][1][3]), NetSnapshot.record_of(pickup)) + assert_eq(NetSnapshot.merge_record(PackedInt32Array(), SimEntity.ARCHETYPE_PROJECTILE, back["records"][2][2], back["records"][2][3]), NetSnapshot.record_of(shot)) + + +func test_a_partial_record_for_an_unknown_entity_is_refused() -> void: + assert_eq(NetSnapshot.merge_record(PackedInt32Array(), SimEntity.ARCHETYPE_PLAYER, 0b1, PackedInt32Array([1, 2, 3])), PackedInt32Array()) + var pickup: PackedInt32Array = PackedInt32Array([SimEntity.ARCHETYPE_PICKUP, 1, 2, 3, 4, 5, 6]) + assert_eq(NetSnapshot.merge_record(pickup, SimEntity.ARCHETYPE_PLAYER, 0b1, PackedInt32Array([1, 2, 3])), PackedInt32Array(), "an ID can't change archetype") + + +func test_truncated_and_corrupt_snapshots_decode_to_nothing_without_errors() -> void: + var sim: Simulation = Simulation.new(3, SimGround.flat()) + var records: Array = [] + for index: int in 5: + var player: SimEntity = _player(sim, Vector3(100.0 + index, 0.0, 100.0)) + records.append([player.id, NetSnapshot.full_mask(SimEntity.ARCHETYPE_PLAYER), NetSnapshot.record_of(player)]) + var bytes: PackedByteArray = NetSnapshot.encode_snapshot({"tick": 5, "own_id": 1}, PackedInt32Array([9]), records) + assert_false(NetSnapshot.decode_snapshot(bytes).is_empty()) + for length: int in bytes.size(): + assert_true(NetSnapshot.decode_snapshot(bytes.slice(0, length)).is_empty(), "cut to %d bytes" % length) + var longer: PackedByteArray = bytes.duplicate() + longer.append(0) + assert_true(NetSnapshot.decode_snapshot(longer).is_empty(), "trailing bytes") + # Every single-byte corruption either decodes to something or to nothing; none may + # raise an engine error, which would fail this test. + var rng: RandomNumberGenerator = RandomNumberGenerator.new() + rng.seed = 77 + for _round: int in 400: + var broken: PackedByteArray = bytes.duplicate() + broken[rng.randi_range(0, broken.size() - 1)] = rng.randi_range(0, 255) + NetSnapshot.decode_snapshot(broken) + for _round: int in 200: + var noise: PackedByteArray = PackedByteArray() + noise.resize(rng.randi_range(0, 80)) + for index: int in noise.size(): + noise[index] = rng.randi_range(0, 255) + if noise.size() > 0: + noise[0] = NetSnapshot.MSG_SNAPSHOT + NetSnapshot.decode_snapshot(noise) + NetSnapshot.decode_input(noise) + NetSnapshot.decode_match(noise) + + +func test_input_round_trips_and_keeps_the_newest_commands() -> void: + var commands: Array[PlayerCommand] = [] + for tick: int in range(50, 62): + commands.append(PlayerCommand.from_input(tick, Vector2(0.5, -1.0), 1.0, 0.2, PlayerCommand.BUTTON_JUMP, 2, 1)) + var bytes: PackedByteArray = NetSnapshot.encode_input(4321, commands) + var back: Dictionary = NetSnapshot.decode_input(bytes) + assert_eq(back["ack"], 4321) + var got: Array[PlayerCommand] = back["commands"] + assert_eq(got.size(), NetSnapshot.MAX_INPUT_COMMANDS) + assert_eq(got[0].tick, 62 - NetSnapshot.MAX_INPUT_COMMANDS) + assert_eq(got[got.size() - 1].tick, 61) + assert_eq(got[0].encode(), commands[commands.size() - NetSnapshot.MAX_INPUT_COMMANDS].encode()) + assert_true(NetSnapshot.decode_input(bytes.slice(0, bytes.size() - 1)).is_empty()) + var lying: PackedByteArray = bytes.duplicate() + lying[5] = 200 + assert_true(NetSnapshot.decode_input(lying).is_empty(), "a count the packet doesn't hold") + assert_eq((NetSnapshot.decode_input(NetSnapshot.encode_input(0, []))["commands"] as Array).size(), 0) + + +func test_match_notice_round_trips_a_63_bit_seed() -> void: + var seed_value: int = 0x7FFFFFFFFFFFFFF1 + var back: Dictionary = NetSnapshot.decode_match(NetSnapshot.encode_match(Match.DROP, 7, seed_value)) + assert_eq(back, {"phase": Match.DROP, "index": 7, "seed": seed_value}) + assert_true(NetSnapshot.decode_match(NetSnapshot.encode_match(99, 7, 1)).is_empty(), "an unknown phase") diff --git a/test/net/test_net_snapshot.gd.uid b/test/net/test_net_snapshot.gd.uid new file mode 100644 index 0000000..133e062 --- /dev/null +++ b/test/net/test_net_snapshot.gd.uid @@ -0,0 +1 @@ +uid://b21v680p10uly diff --git a/test/net/test_snapshot_budget.gd b/test/net/test_snapshot_budget.gd new file mode 100644 index 0000000..62412ab --- /dev/null +++ b/test/net/test_snapshot_budget.gd @@ -0,0 +1,129 @@ +extends TestCase + +## Issue N2's numbers: a whole 24-player match start, every player a client, over a +## link that loses a tenth of its packets and acknowledges late. Measures what each +## client is sent, and checks that nothing outside its interest area is in it. + +## Bytes around every snapshot on the wire: IPv4 and UDP headers, and ENet's protocol +## and command headers for an unreliable, unsequenced send. +const PACKET_OVERHEAD_BYTES: int = 28 + 4 + 8 +const TICKS: int = 2700 +## Snapshots between a snapshot leaving and its acknowledgement arriving. +const ACK_DELAY: int = 2 + + +func test_24_players_stay_under_40_kb_a_second_and_inside_their_interest() -> void: + var loop: ServerLoop = ServerLoop.new(4, 0, Tuning.SOLO_PLAYERS) + var game: NetGame = NetGame.new(loop, NetServer.new()) + var flow: Match = loop.match_flow + var sender: SnapshotSender = game.sender + var receivers: Dictionary = {} + var pending: Dictionary = {} + var handles: Array[int] = flow.participants.duplicate() + assert_eq(handles.size(), Tuning.SOLO_PLAYERS) + for handle: int in handles: + sender.add_client(handle) + receivers[handle] = SnapshotReceiver.new() + pending[handle] = [] + var rng: RandomNumberGenerator = RandomNumberGenerator.new() + rng.seed = 99 + var limit_players: float = Tuning.NET_INTEREST_PLAYERS_METRES + 0.05 + var limit_pickups: float = Tuning.NET_INTEREST_PICKUPS_METRES + 0.05 + var records_seen: int = 0 + var most_entities: int = 0 + var biggest: int = 0 + var send_usec: int = 0 + var rounds: int = 0 + # Per client, the bytes of the busiest second (20 snapshots in a row). + var window: Dictionary = {} + var worst_second: int = 0 + while loop.ticks < TICKS and flow.phase != Match.RESULTS: + loop.run_tick() + if loop.ticks % Tuning.NET_SNAPSHOT_INTERVAL_TICKS != 0: + continue + rounds += 1 + var started: int = Time.get_ticks_usec() + var built: Dictionary = {} + for handle: int in handles: + built[handle] = sender.build(handle, loop.sim, game.viewer_of(handle), loop.ticks, flow.phase) + send_usec += Time.get_ticks_usec() - started + for handle: int in handles: + var bytes: PackedByteArray = built[handle] + var viewer: SimEntity = loop.sim.entities.find(game.viewer_of(handle)) + assert_true(bytes.size() <= Tuning.NET_SNAPSHOT_MAX_BYTES) + biggest = maxi(biggest, bytes.size()) + var recent: Array = window.get_or_add(handle, []) + recent.append(bytes.size() + PACKET_OVERHEAD_BYTES) + if recent.size() > 20: + recent.pop_front() + var second: int = 0 + for size: int in recent: + second += size + worst_second = maxi(worst_second, second) + for entry: Array in NetSnapshot.decode_snapshot(bytes)["records"]: + records_seen += 1 + var entity: SimEntity = loop.sim.entities.find(entry[0]) + if entity == null or viewer == null: + fail("a record for entity %d, which isn't there" % entry[0]) + continue + var away: float = Vector2(entity.position.x - viewer.position.x, entity.position.z - viewer.position.z).length() + var limit: float = limit_pickups if entity.archetype == SimEntity.ARCHETYPE_PICKUP else limit_players + if entity.id != viewer.id and away > limit: + fail("client %d was sent entity %d, %.1f m away (limit %.1f)" % [handle, entity.id, away, limit]) + var receiver: SnapshotReceiver = receivers[handle] + if rng.randf() >= 0.1: + receiver.apply(bytes) + most_entities = maxi(most_entities, receiver.entities.size()) + var queue: Array = pending[handle] + queue.append(receiver.ack_tick()) + if queue.size() > ACK_DELAY: + var ack: int = queue.pop_front() + if rng.randf() >= 0.1: + sender.acknowledge(handle, ack) + assert_true(flow.phase == Match.PLAY or flow.phase == Match.RESULTS, "the run reached play: %s" % flow.phase_name()) + var seconds: float = float(rounds) * Tuning.NET_SNAPSHOT_INTERVAL_TICKS / Tuning.TICK_RATE_HZ + var worst_rate: float = 0.0 + var total: float = 0.0 + var full: int = 0 + var deferred: int = 0 + for handle: int in handles: + var stats: Dictionary = sender.stats(handle) + var rate: float = (float(stats["bytes"]) + float(stats["snapshots"]) * PACKET_OVERHEAD_BYTES) / seconds + worst_rate = maxf(worst_rate, rate) + total += rate + full += int(stats["full"]) + deferred += int(stats["deferred"]) + var receiver: SnapshotReceiver = receivers[handle] + assert_eq(receiver.malformed, 0) + assert_eq(receiver.missing_baseline, 0) + print(" %d clients, %.0f s, 10%% loss: mean %.1f KB/s, worst client %.1f KB/s, worst single second %.1f KB, largest snapshot %d bytes" % [handles.size(), seconds, total / handles.size() / 1000.0, worst_rate / 1000.0, worst_second / 1000.0, biggest]) + print(" %d records sent, %d full snapshots, %d updates deferred, most entities one client held %d, building all 24 took %.2f ms a round" % [records_seen, full, deferred, most_entities, send_usec / 1000.0 / rounds]) + assert_true(worst_rate < 40_000.0, "worst client averaged %.0f bytes a second" % worst_rate) + assert_true(worst_second < 40_000, "worst single second was %d bytes" % worst_second) + assert_true(records_seen > 10_000, "the match really was replicated: %d records" % records_seen) + # Snapshots build on nothing until the first acknowledgement arrives, and again if + # that first one was lost. After that, loss never forces another. + assert_true(full <= handles.size() * (ACK_DELAY + 3), "%d full snapshots" % full) + # With the link quiet again, every client ends with exactly what it may see. + for _round: int in 12: + for _tick: int in Tuning.NET_SNAPSHOT_INTERVAL_TICKS: + loop.run_tick() + if flow.phase == Match.RESULTS: + break + for handle: int in handles: + var receiver: SnapshotReceiver = receivers[handle] + receiver.apply(sender.build(handle, loop.sim, game.viewer_of(handle), loop.ticks, flow.phase)) + sender.acknowledge(handle, receiver.ack_tick()) + var checked: int = 0 + for handle: int in handles: + var receiver: SnapshotReceiver = receivers[handle] + var viewer: SimEntity = loop.sim.entities.find(game.viewer_of(handle)) + var visible: Array[int] = SnapshotSender.interest(loop.sim, viewer) + assert_eq(receiver.entities.size(), visible.size(), "client %d's entity count" % handle) + for entity_id: int in visible: + if receiver.entities.has(entity_id): + checked += 1 + assert_eq(receiver.entities[entity_id], NetSnapshot.record_of(loop.sim.entities.find(entity_id))) + else: + fail("client %d is missing entity %d" % [handle, entity_id]) + assert_true(checked > handles.size()) diff --git a/test/net/test_snapshot_budget.gd.uid b/test/net/test_snapshot_budget.gd.uid new file mode 100644 index 0000000..af36b90 --- /dev/null +++ b/test/net/test_snapshot_budget.gd.uid @@ -0,0 +1 @@ +uid://2npkbpeepyqn diff --git a/test/net/test_snapshot_stream.gd b/test/net/test_snapshot_stream.gd new file mode 100644 index 0000000..92c98ab --- /dev/null +++ b/test/net/test_snapshot_stream.gd @@ -0,0 +1,266 @@ +extends TestCase + +## Issue N2: SnapshotSender and SnapshotReceiver together, over a link that loses, +## delays and repeats packets. Interest management and the size budget. + +const HANDLE: int = 1 +const STEP: int = Tuning.NET_SNAPSHOT_INTERVAL_TICKS + +var sim: Simulation +var viewer: SimEntity +var sender: SnapshotSender +var receiver: SnapshotReceiver +var tick: int = 0 +## Every packet built for the client, delivered or not. +var packets: Array[PackedByteArray] = [] + + +func before_each() -> void: + sim = Simulation.new(5, SimGround.flat()) + viewer = sim.entities.find(sim.add_player()) + viewer.position = Vector3(500.0, 0.0, 500.0) + sender = SnapshotSender.new() + sender.add_client(HANDLE) + receiver = SnapshotReceiver.new() + tick = 0 + packets.clear() + + +func _other(at: Vector3) -> SimEntity: + var player: SimEntity = sim.entities.find(sim.add_player()) + player.position = at + return player + + +## Builds the next snapshot. Delivers it and returns the acknowledgement unless told not to. +func _send(deliver: bool = true, acknowledge: bool = true) -> PackedByteArray: + tick += STEP + var bytes: PackedByteArray = sender.build(HANDLE, sim, viewer.id, tick, Match.PLAY) + packets.append(bytes) + assert_true(bytes.size() <= Tuning.NET_SNAPSHOT_MAX_BYTES, "snapshot of %d bytes" % bytes.size()) + if deliver: + receiver.apply(bytes) + if acknowledge: + sender.acknowledge(HANDLE, receiver.ack_tick()) + return bytes + + +## Fails unless the client's picture is exactly what it may see, as it is now. +func _assert_in_step(context: String) -> void: + var visible: Array[int] = SnapshotSender.interest(sim, viewer) + assert_eq(receiver.entities.size(), visible.size(), "%s: entity count" % context) + for entity_id: int in visible: + if not receiver.entities.has(entity_id): + fail("%s: entity %d is missing" % [context, entity_id]) + continue + assert_eq(receiver.entities[entity_id], NetSnapshot.record_of(sim.entities.find(entity_id)), "%s: entity %d" % [context, entity_id]) + + +func test_first_snapshot_is_full_and_later_ones_carry_only_changes() -> void: + var other: SimEntity = _other(Vector3(520.0, 0.0, 500.0)) + var first: PackedByteArray = _send() + assert_eq(NetSnapshot.decode_snapshot(first)["baseline"], 0) + assert_eq(receiver.own_id, viewer.id) + _assert_in_step("first") + var quiet: Dictionary = NetSnapshot.decode_snapshot(_send()) + assert_eq(quiet["baseline"], tick - STEP) + assert_eq((quiet["records"] as Array).size(), 0, "nothing moved") + other.position.x += 1.0 + var moved: Dictionary = NetSnapshot.decode_snapshot(_send()) + assert_eq((moved["records"] as Array).size(), 1) + assert_eq(moved["records"][0][2], 0b1, "only the position group") + _assert_in_step("moved") + assert_eq(sender.stats(HANDLE)["full"], 1) + + +func test_header_carries_the_tide_the_clock_and_the_command_echo() -> void: + sim.tide = Tide.new(0) + sim.sea_level = 3.25 + sim.tick = 77 + sender.note_command(HANDLE, 75) + sender.note_command(HANDLE, 60) + _send() + assert_near(receiver.sea_level, 3.25, 0.001) + assert_eq(receiver.sim_tick, 77) + assert_eq(receiver.command_tick, 75) + assert_eq(receiver.match_phase, Match.PLAY) + + +func test_the_client_recovers_from_lost_snapshots_and_lost_acknowledgements() -> void: + var others: Array[SimEntity] = [] + for index: int in 12: + others.append(_other(Vector3(480.0 + index * 4.0, 0.0, 510.0))) + var rng: RandomNumberGenerator = RandomNumberGenerator.new() + rng.seed = 2026 + var delivered: int = 0 + for round: int in 400: + for other: SimEntity in others: + if rng.randf() < 0.6: + other.position += Vector3(rng.randf_range(-1.0, 1.0), 0.0, rng.randf_range(-1.0, 1.0)) + other.yaw = rng.randi_range(0, 65535) + if rng.randf() < 0.05: + other.health = rng.randi_range(1, 100) + if round == 150: + sim.remove_player(others.pop_back().id) + var arrives: bool = rng.randf() >= 0.4 + _send(arrives, rng.randf() >= 0.4) + if arrives: + delivered += 1 + _assert_in_step("round %d" % round) + assert_true(delivered > 150 and delivered < 300, "the link lost a real share: %d of 400 arrived" % delivered) + assert_eq(receiver.missing_baseline, 0, "a delta never builds on something the client didn't confirm") + assert_eq(receiver.malformed, 0) + assert_eq(sender.stats(HANDLE)["full"], 1, "loss alone never needs a second full snapshot") + + +func test_duplicates_and_stale_snapshots_are_ignored() -> void: + var other: SimEntity = _other(Vector3(520.0, 0.0, 500.0)) + var old: PackedByteArray = _send() + other.position.x += 5.0 + var newer: PackedByteArray = _send() + assert_false(receiver.apply(old), "an older snapshot arriving late") + assert_false(receiver.apply(newer), "the same snapshot twice") + assert_eq(receiver.ignored_old, 2) + _assert_in_step("after repeats") + + +func test_a_long_silence_falls_back_to_a_full_snapshot() -> void: + var other: SimEntity = _other(Vector3(520.0, 0.0, 500.0)) + _send() + # Longer than the server remembers: everything it could build on is forgotten. + for _round: int in Tuning.NET_SNAPSHOT_HISTORY + 5: + other.position.x += 0.1 + _send(false, false) + var bytes: PackedByteArray = _send() + assert_eq(NetSnapshot.decode_snapshot(bytes)["baseline"], 0) + _assert_in_step("after the silence") + assert_true(sender.stats(HANDLE)["full"] >= 2) + + +func test_a_client_that_lost_its_baseline_asks_again_and_recovers() -> void: + _other(Vector3(520.0, 0.0, 500.0)) + _send() + _send() + # A client that restarted its picture: it holds nothing the server builds on. + receiver = SnapshotReceiver.new() + _send() + assert_eq(receiver.missing_baseline, 1) + assert_eq(receiver.ack_tick(), 0) + assert_eq(receiver.entities.size(), 0) + _send() + _assert_in_step("after asking again") + + +func test_no_data_about_entities_outside_the_interest_area() -> void: + var range: float = Tuning.NET_INTEREST_PLAYERS_METRES + var near: SimEntity = _other(viewer.position + Vector3(range - 5.0, 0.0, 0.0)) + var far: SimEntity = _other(viewer.position + Vector3(range + 5.0, 0.0, 0.0)) + var far_item: int = Inventory.spawn_pickup(sim, viewer.position + Vector3(0.0, 0.0, Tuning.NET_INTEREST_PICKUPS_METRES + 3.0), 1, 1) + var near_item: int = Inventory.spawn_pickup(sim, viewer.position + Vector3(0.0, 0.0, Tuning.NET_INTEREST_PICKUPS_METRES - 3.0), 1, 1) + for _round: int in 10: + far.position.z += 1.0 + far.health -= 1 + _send() + assert_true(receiver.entities.has(near.id) and receiver.entities.has(near_item)) + for bytes: PackedByteArray in packets: + var snapshot: Dictionary = NetSnapshot.decode_snapshot(bytes) + for entry: Array in snapshot["records"]: + assert_true(entry[0] != far.id and entry[0] != far_item, "a record for an entity out of range") + assert_false((snapshot["removed"] as PackedInt32Array).has(far.id), "even its ID stays unsaid") + # It walks into range: now the client hears of it, in full. + far.position = viewer.position + Vector3(range - 1.0, 0.0, 0.0) + _send() + assert_true(receiver.entities.has(far.id)) + assert_eq(Array(receiver.added), [far.id]) + # And out again: it is removed, and nothing more is said about it. + far.position = viewer.position + Vector3(range + 1.0, 0.0, 0.0) + _send() + assert_false(receiver.entities.has(far.id)) + assert_eq(Array(receiver.removed), [far.id]) + var before: int = packets.size() + for _round: int in 5: + far.position.z -= 1.0 + _send() + for index: int in range(before, packets.size()): + assert_eq((NetSnapshot.decode_snapshot(packets[index])["records"] as Array).size(), 0) + _assert_in_step("at the end") + + +func test_height_does_not_hide_a_player_overhead() -> void: + var above: SimEntity = _other(viewer.position + Vector3(10.0, Tuning.BARGE_ALTITUDE, 0.0)) + _send() + assert_true(receiver.entities.has(above.id)) + + +func test_a_crowd_is_spread_over_several_snapshots_nearest_first() -> void: + var ids: Array[int] = [] + for index: int in 600: + var angle: float = index * 0.37 + var reach: float = 2.0 + float(index) / 600.0 * (Tuning.NET_INTEREST_PICKUPS_METRES - 4.0) + ids.append(Inventory.spawn_pickup(sim, viewer.position + Vector3(cos(angle) * reach, 0.0, sin(angle) * reach), 1, 1)) + var first: PackedByteArray = _send() + assert_true(first.size() > Tuning.NET_SNAPSHOT_MAX_BYTES - 16, "the first snapshot is full to the brim") + assert_true(receiver.entities.has(viewer.id), "the client's own player always travels") + var got: int = receiver.entities.size() - 1 + assert_true(got > 50 and got < 600, "%d of 600 fitted" % got) + var furthest_got: float = 0.0 + var nearest_missing: float = 1.0e9 + for entity_id: int in ids: + var distance: float = sim.entities.find(entity_id).position.distance_to(viewer.position) + if receiver.entities.has(entity_id): + furthest_got = maxf(furthest_got, distance) + else: + nearest_missing = minf(nearest_missing, distance) + assert_true(furthest_got <= nearest_missing + 0.1, "nearest first: got out to %.1f m, missing from %.1f m" % [furthest_got, nearest_missing]) + for _round: int in 12: + _send() + _assert_in_step("after a dozen snapshots") + assert_true(sender.stats(HANDLE)["deferred"] > 0) + + +func test_a_distant_entity_is_not_starved_by_a_busy_foreground() -> void: + # More near entities changing every snapshot than one snapshot can hold. + var busy: Array[SimEntity] = [] + for index: int in 200: + busy.append(_other(viewer.position + Vector3(5.0 + index * 0.5, 0.0, 3.0))) + var distant: SimEntity = _other(viewer.position + Vector3(0.0, 0.0, Tuning.NET_INTEREST_PLAYERS_METRES - 10.0)) + for _round: int in 30: + _send() + var seen: int = 0 + for round: int in 60: + for other: SimEntity in busy: + other.position.z += 0.25 + other.yaw = (other.yaw + 500) % 65536 + distant.position.x += 0.5 + var before: PackedInt32Array = receiver.entities.get(distant.id, PackedInt32Array()) + _send() + if receiver.entities.get(distant.id, PackedInt32Array()) != before: + seen += 1 + assert_true(sender.stats(HANDLE)["deferred"] > 0, "the snapshots really were over full") + assert_true(seen >= 3, "the distant player was updated %d times in 60 snapshots" % seen) + + +func test_a_new_match_starts_from_nothing() -> void: + _other(Vector3(520.0, 0.0, 500.0)) + _send() + _send() + # A new simulation reuses entity IDs for different things. + sender.reset() + sim = Simulation.new(6, SimGround.flat()) + viewer = sim.entities.find(sim.add_player()) + viewer.position = Vector3(100.0, 0.0, 100.0) + Inventory.spawn_pickup(sim, Vector3(101.0, 0.0, 100.0), 1, 1) + var bytes: PackedByteArray = _send() + assert_eq(NetSnapshot.decode_snapshot(bytes)["baseline"], 0) + _assert_in_step("new match") + assert_eq(receiver.malformed, 0) + + +func test_no_viewer_means_an_empty_snapshot() -> void: + _other(Vector3(520.0, 0.0, 500.0)) + tick += STEP + var bytes: PackedByteArray = sender.build(HANDLE, sim, 0, tick, Match.LOBBY) + assert_true(receiver.apply(bytes)) + assert_eq(receiver.entities.size(), 0) + assert_eq(receiver.own_id, 0) + assert_eq(receiver.match_phase, Match.LOBBY) diff --git a/test/net/test_snapshot_stream.gd.uid b/test/net/test_snapshot_stream.gd.uid new file mode 100644 index 0000000..522554f --- /dev/null +++ b/test/net/test_snapshot_stream.gd.uid @@ -0,0 +1 @@ +uid://cfvw1mabf2tdh diff --git a/test/server/test_net_game.gd b/test/server/test_net_game.gd new file mode 100644 index 0000000..149edce --- /dev/null +++ b/test/server/test_net_game.gd @@ -0,0 +1,188 @@ +extends TestCase + +## Issue N2 end to end: a real NetServer, a ServerLoop and NetClients over loopback +## UDP. People join the lobby, the match starts, snapshots arrive and inputs steer. + +const HOST: String = "127.0.0.1" + +var loop: ServerLoop +var net: NetServer +var game: NetGame +var clients: Array[NetClient] = [] +var receivers: Array[SnapshotReceiver] = [] +## Per client: every MATCH notice it was sent. +var notices: Array[Array] = [] +## Per client: the yaw it steers towards, in command steps. +var steer: Array[int] = [] +var joined: Array[Dictionary] = [] +var left: Array[Dictionary] = [] + + +func before_each() -> void: + var rules: MatchRules = MatchRules.new() + rules.max_players = 4 + rules.min_players = 2 + rules.lobby_ticks = 6 + rules.countdown_ticks = 6 + rules.results_ticks = 6 + loop = ServerLoop.new(31, 0, 2, rules) + net = NetServer.new() + assert_eq(net.listen(0, HOST), OK) + game = NetGame.new(loop, net) + clients.clear() + receivers.clear() + notices.clear() + steer.clear() + joined.clear() + left.clear() + + +func after_each() -> void: + for client: NetClient in clients: + client.leave() + net.close() + + +func _client(wanted: String) -> NetClient: + var client: NetClient = NetClient.new() + clients.append(client) + receivers.append(SnapshotReceiver.new()) + notices.append([]) + steer.append(0) + assert_eq(client.connect_to(HOST, net.port(), wanted), OK) + return client + + +## Runs the server and every client, a tick per round, until `done` or the time is up. +func _pump(done: Callable, limit_msec: int = 8000) -> bool: + var started: int = Time.get_ticks_msec() + while Time.get_ticks_msec() - started < limit_msec: + game.poll() + for event: Dictionary in game.events: + (joined if event["type"] == "joined" else left).append(event) + loop.run_tick() + game.after_tick() + for index: int in clients.size(): + var client: NetClient = clients[index] + var receiver: SnapshotReceiver = receivers[index] + client.poll() + for packet: Dictionary in client.packets: + var bytes: PackedByteArray = packet["bytes"] + if packet["channel"] == NetProtocol.CHANNEL_CONTROL: + var notice: Dictionary = NetSnapshot.decode_match(bytes) + if not notice.is_empty(): + notices[index].append(notice) + elif receiver.apply(bytes): + pass + if client.state == NetClient.JOINED: + var commands: Array[PlayerCommand] = [] + for ahead: int in range(1, Tuning.NET_INPUT_REDUNDANCY + 1): + var command: PlayerCommand = PlayerCommand.new() + command.tick = receiver.sim_tick + ahead + command.yaw = steer[index] + commands.append(command) + client.send(NetProtocol.CHANNEL_STATE, NetSnapshot.encode_input(receiver.ack_tick(), commands), false) + if done.call(): + return true + OS.delay_msec(1) + return bool(done.call()) + + +func test_two_people_join_play_and_are_sent_the_match() -> void: + var mara: NetClient = _client("Mara") + var odo: NetClient = _client("Odo") + steer[0] = 12345 + steer[1] = 54321 + assert_true(_pump(func() -> bool: return loop.match_flow.in_match() and receivers[0].own_id != 0 and receivers[1].own_id != 0), "the match started and both were told") + var flow: Match = loop.match_flow + assert_eq(joined.size(), 2) + assert_eq(flow.participants.size(), 4, "two people and two bots") + assert_eq(receivers[0].own_id, flow.entity_of[mara.handle]) + assert_eq(receivers[1].own_id, flow.entity_of[odo.handle]) + # Each was told the seed and every phase on the way in. + for index: int in 2: + var phases: Array[int] = [] + for notice: Dictionary in notices[index]: + phases.append(notice["phase"]) + assert_eq(notice["seed"], flow.current_seed()) + assert_eq(phases.slice(phases.size() - 2), [Match.COUNTDOWN, Match.DROP], "client %d's notices: %s" % [index, phases]) + # Everyone starts aboard the barge together, so each sees all four players. + assert_true(_pump(func() -> bool: return receivers[0].entities.size() >= 4 and receivers[1].entities.size() >= 4)) + # Inputs arrive and steer: the server took commands, and each player faces its way. + assert_true(_pump(func() -> bool: + var mine: PackedInt32Array = receivers[0].entities.get(receivers[0].own_id, PackedInt32Array()) + var theirs: PackedInt32Array = receivers[0].entities.get(receivers[1].own_id, PackedInt32Array()) + return not mine.is_empty() and not theirs.is_empty() and mine[NetSnapshot.R_YAW] == 12345 and theirs[NetSnapshot.R_YAW] == 54321 + ), "both yaws reached Mara's picture") + assert_true(receivers[0].command_tick > 0, "the server echoes the command tick it took") + assert_eq(game.bad_inputs, 0) + assert_true(receivers[0].applied > 5) + assert_eq(receivers[0].malformed + receivers[1].malformed, 0) + assert_true(game.sender.stats(mara.handle)["bytes"] > 0) + + +func test_commands_repeated_in_packets_reach_the_simulation_once() -> void: + var mara: NetClient = _client("Mara") + _client("Odo") + assert_true(_pump(func() -> bool: return loop.match_flow.phase == Match.COUNTDOWN)) + var log: CommandLog = loop.record_commands() + assert_true(_pump(func() -> bool: return loop.match_flow.in_match() and loop.sim.tick > 40)) + var ticks_seen: Dictionary = {} + var from_mara: int = 0 + for index: int in log.count(): + if log.handle_of(index) != mara.handle: + continue + from_mara += 1 + var command_tick: int = log.command_of(index).tick + assert_false(ticks_seen.has(command_tick), "tick %d submitted twice" % command_tick) + ticks_seen[command_tick] = true + assert_true(from_mara > 10, "%d commands from Mara" % from_mara) + + +func test_leaving_takes_the_player_out_of_the_match() -> void: + var mara: NetClient = _client("Mara") + var odo: NetClient = _client("Odo") + assert_true(_pump(func() -> bool: return loop.match_flow.in_match() and receivers[1].own_id != 0)) + var handle: int = mara.handle + var entity_id: int = loop.match_flow.entity_of[handle] + mara.leave() + assert_true(_pump(func() -> bool: return left.size() == 1)) + assert_eq(left[0]["handle"], handle) + assert_false(loop.match_flow.participants.has(handle)) + assert_false(game.sender.has_client(handle)) + # Odo's picture loses her within a few snapshots. + assert_true(_pump(func() -> bool: return not receivers[1].entities.has(entity_id))) + assert_eq(odo.state, NetClient.JOINED) + + +func test_a_full_match_turns_the_next_person_away() -> void: + _client("Mara") + _client("Odo") + assert_true(_pump(func() -> bool: return loop.match_flow.in_match())) + # Bots filled the other two places, so there is no room until this match ends. + var late: NetClient = _client("Late") + assert_true(_pump(func() -> bool: return late.state == NetClient.CLOSED)) + assert_eq(late.close_reason, NetProtocol.REASON_FULL) + assert_eq(loop.match_flow.participants.size(), 4) + assert_eq(net.player_count(), 2) + + +func test_junk_on_the_game_channel_is_counted_and_changes_nothing() -> void: + var mara: NetClient = _client("Mara") + _client("Odo") + assert_true(_pump(func() -> bool: return loop.match_flow.in_match())) + var rng: RandomNumberGenerator = RandomNumberGenerator.new() + rng.seed = 5 + for _packet: int in 20: + var bytes: PackedByteArray = PackedByteArray() + bytes.resize(rng.randi_range(1, 200)) + for index: int in bytes.size(): + bytes[index] = rng.randi_range(0, 255) + bytes[0] = NetSnapshot.MSG_INPUT if rng.randf() < 0.5 else rng.randi_range(0, 255) + # Never a well-formed length for an INPUT. + if (bytes.size() - 6) % PlayerCommand.ENCODED_SIZE == 0: + bytes.append(0) + mara.send(NetProtocol.CHANNEL_STATE, bytes, true) + assert_true(_pump(func() -> bool: return game.bad_inputs >= 20), "%d counted" % game.bad_inputs) + assert_eq(mara.state, NetClient.JOINED) + assert_true(loop.match_flow.in_match()) diff --git a/test/server/test_net_game.gd.uid b/test/server/test_net_game.gd.uid new file mode 100644 index 0000000..3c8841b --- /dev/null +++ b/test/server/test_net_game.gd.uid @@ -0,0 +1 @@ +uid://ba86p3w67wje5 diff --git a/test/server/test_server_config.gd b/test/server/test_server_config.gd index 1afb72d..4ef752f 100644 --- a/test/server/test_server_config.gd +++ b/test/server/test_server_config.gd @@ -40,6 +40,15 @@ func test_all_options() -> void: assert_eq(config.to_dictionary()["bot_difficulty"], "hard") +func test_listen_is_off_by_default_and_refuses_idle_slots() -> void: + assert_false(_parse(["--server"]).listen) + var config: ServerConfig = _parse(["--server", "--listen", "--port=4001", "--bots=24"]) + assert_true(config.is_valid(), str(config.errors)) + assert_true(config.listen) + assert_eq(config.to_dictionary()["listen"], true) + assert_false(_parse(["--server", "--listen", "--players=2"]).is_valid()) + + func test_bot_difficulty_defaults_to_mixed_and_rejects_unknown_names() -> void: assert_eq(_parse(["--server"]).bot_difficulty, -1) assert_eq(_parse(["--server", "--bot-difficulty=mixed"]).bot_difficulty, -1) diff --git a/tools/smoke.sh b/tools/smoke.sh index 043f302..aacbcc5 100755 --- a/tools/smoke.sh +++ b/tools/smoke.sh @@ -4,7 +4,8 @@ # - exits 0 and logs start, stats and stop lines that all carry the seed, # - produces the same state hash when run again with the same seed, # - exits non-zero when a script error happens (a deliberate --fault), -# - exits non-zero on a bad command line. +# - exits non-zero on a bad command line, +# - listens on a UDP port when asked to. # # tools/smoke.sh unthrottled, so it finishes in seconds # SMOKE_REALTIME=1 tools/smoke.sh at the real 30 Hz tick rate (about 34 seconds) @@ -69,7 +70,14 @@ run --bots=99 --nonsense [[ "$code" -eq 2 ]] || fail "a bad command line exited $code, expected 2" grep -q '"event":"config_error"' <<<"$output" || fail "a bad command line logged no config_error" +# A listening server opens its port, waits in the lobby with nobody there, and stops cleanly. +port=$((20000 + RANDOM % 20000)) +run --listen --port="$port" --seed="$seed" --bots=4 --ticks=120 --unthrottled +[[ "$code" -eq 0 ]] || fail "a listening server exited $code" +[[ "$(field start listening)" == "true" ]] || fail "a listening server didn't say it was listening" +[[ "$(field start match_phase)" == "lobby" ]] || fail "a listening server didn't start in the lobby" + if [[ "$status" -eq 0 ]]; then - echo "smoke: ok (clean run, reproducible hash, script error exits 1, bad config exits 2)" + echo "smoke: ok (clean run, reproducible hash, script error exits 1, bad config exits 2, listens)" fi exit "$status"