diff --git a/CLAUDE.md b/CLAUDE.md index d6f8554..9fa88bb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -553,6 +553,28 @@ arithmetic (no regression outside vtmgr). Applied so far in `assets/disk0/tvdos/bin/taut.js` and `assets/disk0/hopper/include/aa.mjs` (used by `bb.js`). Any future direct-VRAM app needs the same one-line `vaddr`. +### Raw-keyboard apps must grab (the `con.grabRawKeyboard` pattern) + +Fullscreen apps that poll the **raw key snapshot** (`sys.poke(-40,1)` then +`sys.peek(-41..-48)`) directly — e.g. the DOOM port's `i_input.mjs` — bypass the +pane input ring entirely. But the dispatcher keeps the cooked collector (`-39`) +on and drains typed chars into the *active* pane's ring every frame. While such +an app is the active pane, every keystroke piles into a ring it never reads, and +floods its parent shell the instant the app exits (no bug outside vtmgr, where +`-39` is off while a raw app runs). Fix: the pane bootstrap exposes +`con.grabRawKeyboard()` / `con.releaseRawKeyboard()` (write the active VT number +into `CTRL+CTRL_RAW_GRAB_VT`); while the active pane holds the grab the +dispatcher discards cooked chars and keeps that pane's ring flushed. `con.getch` +self-heals a grab leaked by a crashed app (a cooked reader isn't a grabber). +A raw-input app feature-detects (`typeof con.grabRawKeyboard === "function"`) and +grabs/releases around its fullscreen session — DOOM does it in +`i_video.mjs` `I_InitGraphics`/`I_ShutdownGraphics` (covers every fullscreen +mode; shutdown runs in `wadplayer.js`'s `finally`). Complementary: such an app's +poll should also no-op when it's *not* the active VT (compare `VT_CTRL_ADDR` +byte 0 to `VT_NUM`) so a backgrounded app doesn't eat the foreground console's +input — DOOM's `I_PollKeys` does this. Any future raw-key app under vtmgr needs +both. + ### Files - New: `assets/disk0/tvdos/VTMGR.SYS` (dispatcher + per-pane bootstrap) @@ -565,6 +587,10 @@ arithmetic (no regression outside vtmgr). Applied so far in - `assets/disk0/AUTOEXEC.BAT`: per-console launch (Korean IME + `command -fancy`) - `assets/disk0/tvdos/bin/taut.js`, `assets/disk0/hopper/include/aa.mjs`: `vaddr` VT-aware direct-VRAM addressing +- `assets/disk0/tvdos/VTMGR.SYS`: `CTRL_RAW_GRAB_VT` flag + + `con.grabRawKeyboard`/`releaseRawKeyboard` (raw-keyboard apps); dispatcher + drain honours it. DOOM consumer: `assets/disk0/home/doom/i_video.mjs` + (grab/release) + `i_input.mjs` (active-VT poll guard) ### Gotcha: injectIntChk vs. embedded source diff --git a/assets/disk0/tvdos/VTMGR.SYS b/assets/disk0/tvdos/VTMGR.SYS index 2e75993..fcf3538 100644 --- a/assets/disk0/tvdos/VTMGR.SYS +++ b/assets/disk0/tvdos/VTMGR.SYS @@ -17,7 +17,10 @@ // +1 switch_request u8 (0 = none, 1..6 = target; set by chvt, cleared by dispatcher) // +2 debounce_held u8 // +3 vt_spawned_bits u8 (bit n-1 set if VT n is alive) -// +4..63 reserved +// +4 raw_grab_vt u8 (0 = none; else VT n holding a raw-keyboard grab, +// i.e. a fullscreen app reading -41..-48 directly, e.g. DOOM. While set +// and == active_vt, the dispatcher stops feeding that pane's ring.) +// +5..63 reserved // VT block (× MAX_VT) starting at base + 64, each VT_BLOCK_SIZE bytes // +0..7 reserved (cursor & color state lives inside text plane itself) // +8 queue_head u8 (next-read index) @@ -38,6 +41,7 @@ const CTRL_ACTIVE_VT = 0 const CTRL_SWITCH_REQUEST = 1 const CTRL_DEBOUNCE_HELD = 2 const CTRL_SPAWNED_BITS = 3 +const CTRL_RAW_GRAB_VT = 4 const GPU_TEXTAREA_OFFSET = 253950 const TEXT_COLS = 80 @@ -381,6 +385,8 @@ con.prnch = function(c) { // keyboard MMIO — that's the dispatcher's exclusive territory. Cooperative // gate on active_vt keeps background panes parked when they call getch. +const RAW_GRAB_ADDR = CTRL + ${CTRL_RAW_GRAB_VT} + function queuePop() { let head = sys.peek(QUEUE_HEAD_ADDR) let tail = sys.peek(QUEUE_TAIL_ADDR) @@ -390,6 +396,10 @@ function queuePop() { return b } con.getch = function() { + // Reading cooked input means we are NOT a raw-keyboard grabber; drop any + // stale grab this VT left set (e.g. a fullscreen app that crashed without + // releasing) so the dispatcher resumes feeding our ring. + if (sys.peek(RAW_GRAB_ADDR) === VT_NUM) sys.poke(RAW_GRAB_ADDR, 0) while (true) { if (sys.peek(ACTIVE_VT_ADDR) === VT_NUM) { let k = queuePop() @@ -398,6 +408,18 @@ con.getch = function() { sys.sleep(20) } } +// A fullscreen app that reads the raw keyboard snapshot (-41..-48) directly, +// bypassing this ring, must grab so the dispatcher stops piling cooked chars +// into a ring it never drains. Flush any prior type-ahead on grab; the +// dispatcher keeps the ring empty while held. Release on exit (or the next +// con.getch self-heals a grab leaked by a crashed app). +con.grabRawKeyboard = function() { + sys.poke(RAW_GRAB_ADDR, VT_NUM) + sys.poke(QUEUE_HEAD_ADDR, sys.peek(QUEUE_TAIL_ADDR)) +} +con.releaseRawKeyboard = function() { + if (sys.peek(RAW_GRAB_ADDR) === VT_NUM) sys.poke(RAW_GRAB_ADDR, 0) +} con.hitterminate = function() { return false } con.hiteof = function() { return false } con.resetkeybuf = function() { sys.poke(QUEUE_HEAD_ADDR, sys.peek(QUEUE_TAIL_ADDR)) } @@ -555,11 +577,23 @@ while (running) { // keyboardBuffer, so doing it every frame would drop chars typed last frame. if (sys.peek(-39) === 0) sys.poke(-39, 1) - // drain typed chars into the active pane's queue + // drain typed chars into the active pane's queue — UNLESS that pane holds + // a raw-keyboard grab (a fullscreen app reading -41..-48 directly, e.g. + // DOOM). Such an app never reads its ring, so cooked chars would pile up + // there and flood its shell the instant the app exits. We still drain the + // cooked buffer (must read -38 to clear the -50 count) but discard the + // chars, and keep the grabbing pane's ring flushed so nothing surfaces + // later. Alt-N is unaffected — it reads the raw snapshot above. + const rawGrab = sys.peek(CTRL + CTRL_RAW_GRAB_VT) + const feedRing = rawGrab !== active while (sys.peek(-50) !== 0) { let k = sys.peek(-38) if (k < 0) k += 256 - queuePush(active, k) + if (feedRing) queuePush(active, k) + } + if (!feedRing) { + const qb = vtBlockAddr(active) + sys.poke(qb + 8, sys.peek(qb + 9)) // head = tail: flush stale type-ahead } sys.sleep(33) diff --git a/assets/disk0/tvdos/include/taud.mjs b/assets/disk0/tvdos/include/taud.mjs index c0200e4..fc51bd0 100644 --- a/assets/disk0/tvdos/include/taud.mjs +++ b/assets/disk0/tvdos/include/taud.mjs @@ -9,6 +9,15 @@ const TAUD_MAGIC = [0x1F,0x54,0x53,0x56,0x4D,0x61,0x75,0x64] // \x1F TSVMaud const TAUD_VERSION = 1 const TAUD_HEADER_SIZE = 32 // magic(8) + version(1) + numSongs(1) + compSize(4) + projOff(4) + sig(14) +// Container kind = top two bits of the version byte (terranmon.txt:3342-3401). +// 00 → full .taud (sample+inst image + song table + patterns) +// 10 → .tsii (sample+inst image only; numSongs = 0, no song table) +// 11 → .tpif (patterns only; sample+inst compSize = 0 — section absent — +// instruments come from a previously-loaded .tsii) +const TAUD_KIND_MASK = 0xC0 +const TAUD_KIND_FULL = 0x00 +const TAUD_KIND_SAMPLEINST = 0x80 +const TAUD_KIND_PATTERN = 0xC0 const TAUD_SONG_ENTRY = 32 // see encodeSongEntry / decodeSongEntry below // Sample+instrument image: 8 MB sample pool (banked, 16 × 512 K) + 64 K instrument bin = 8256 kB total. // (terranmon.txt:1985-1997, 2533-2564 — bank-switched via MMIO 46.) @@ -46,12 +55,18 @@ function _pokeU32LE(ptr, off, v) { // ── uploadTaudFile ────────────────────────────────────────────────────────── /** - * Load one song from a Taud file into the tracker hardware and configure the - * given playhead ready to play. + * Load a Taud container into the tracker hardware. Handles all three kinds + * (terranmon.txt:3342-3401), distinguished by the top two bits of the version + * byte: + * - full .taud (00): uploads the sample+instrument image AND loads one song. + * - .tsii (10): uploads the sample+instrument image ONLY (the shared bank for a + * collection of .tpif files). songIndex / playhead are ignored. + * - .tpif (11): loads one song's patterns ONLY, leaving the resident + * sample+instrument bank untouched — load the companion .tsii FIRST. * * @param inFile Full path with drive letter, e.g. "A:/music/song.taud" - * @param songIndex 0-based index of the song in the SONG TABLE - * @param playhead Playhead number (0-3) to configure + * @param songIndex 0-based index of the song in the SONG TABLE (ignored for .tsii) + * @param playhead Playhead number (0-3) to configure (ignored for .tsii) */ function uploadTaudFile(inFile, songIndex, playhead) { const drive = inFile[0].toUpperCase() @@ -92,21 +107,32 @@ function uploadTaudFile(inFile, songIndex, playhead) { pos += 14 // signature // pos == 32 == TAUD_HEADER_SIZE - if (songIndex < 0 || songIndex >= numSongs) { - sys.free(filePtr) - throw Error("taud: songIndex " + songIndex + " out of range (numSongs=" + numSongs + ")") - } + const kind = version & TAUD_KIND_MASK + const isSampleInst = (kind === TAUD_KIND_SAMPLEINST) // .tsii: instruments only + const isPattern = (kind === TAUD_KIND_PATTERN) // .tpif: patterns only // -- 4. Decompress and upload sample+instrument bin ----------------------- // The decompressed image is 8256 kB (8 MB samples bank-major + 64 K instruments) // which exceeds the 8 MB user-space cap, so we route through a hardware helper // that decompresses straight into the adapter's native sample/instrument // storage instead of staging a buffer in user memory. - audio.uploadSampleInstBlob(filePtr + pos, compressedSize) - audio.setSampleBank(0) - pos += compressedSize + // Skipped for .tpif — its sample+inst section is absent (compSize = 0) and the + // resident bank (from a previously-loaded .tsii) must be left intact. + if (!isPattern) { + audio.uploadSampleInstBlob(filePtr + pos, compressedSize) + audio.setSampleBank(0) + pos += compressedSize + } - // -- 5. Parse song-table entry for the requested song -------------------- + // -- 5. Song table → patterns → cues → playhead (full .taud / .tpif only) -- + // A .tsii carries no song table (numSongs = 0); it stops after the bank + Ixmp. + if (!isSampleInst) { + if (songIndex < 0 || songIndex >= numSongs) { + sys.free(filePtr) + throw Error("taud: songIndex " + songIndex + " out of range (numSongs=" + numSongs + ")") + } + + // -- 5a. Parse song-table entry for the requested song ------------------- let entryOff = pos + songIndex * TAUD_SONG_ENTRY let songOffset = _peekU32LE(filePtr, entryOff) let numVoices = sys.peek(filePtr + entryOff + 4) & 0xFF @@ -156,8 +182,11 @@ function uploadTaudFile(inFile, songIndex, playhead) { audio.setTrackerMixerFlags(playhead, mixerflags) audio.setSongGlobalVolume(playhead, songGlobalVolume) audio.setSongMixingVolume(playhead, songMixingVolume) + } // end !isSampleInst (song table / patterns / cues / playhead) // -- 9. Project Data — walk Ixmp blocks for multi-sample instruments ----- + // Runs for every kind: a .tsii carries its instruments' Ixmp patches here; a + // .tpif carries only p/s blocks (sMet) and contributes no patches. // Terranmon spec: Project Data starts at `projOff` (zero = absent), magic is // \x1ETaudPrJ + 8 reserved bytes, then a stream of FourCC + Uint32-length // sections. We only consume "Ixmp" here; other sections (PNam, INam, sMet, diff --git a/midi2taud.py b/midi2taud.py index 78b604b..44de519 100644 --- a/midi2taud.py +++ b/midi2taud.py @@ -8,6 +8,20 @@ Usage: [--bend-epsilon CENTS] [--drum-keyoff] [-v] [--no-project-data] + # Batch / directory mode (terranmon.txt:3342-3401): + python3 midi2taud.py midi_dir/ soundfont.sf2 [out_dir/] + + When the first argument is a DIRECTORY, every .mid/.midi inside it is compiled + against the one SoundFont into the split Taud format: a single shared Sample and + Instrument Image (.tsii) holding the instrument bank for all the songs, + plus one Pattern Image (.tpif) per MIDI carrying just that song's patterns. + A .tpif is played by first loading its companion .tsii. The shared bank spans the + UNION of every song's instruments, so the 8 MB sample / 255 slot budgets are shared + too (overflow degrades exactly as in single-file mode). Instrument fadeouts encode + SF2 release times in seconds but the engine fades per song-tick (rate ∝ BPM), so the + shared image targets the mean of the songs' initial tempos; pass --fadeout for a + tempo-independent step. Output directory defaults to the input directory. + Behaviour (per midi2taud.md): * Pitch bends are preserved as much as possible. A note starting under a non-zero bend triggers directly at the bent 4096-TET pitch (Taud notes @@ -85,6 +99,7 @@ import sys from taud_common import ( set_verbose, vprint, TAUD_MAGIC, TAUD_VERSION, TAUD_HEADER_SIZE, TAUD_SONG_ENTRY, + TAUD_KIND_FULL, TAUD_KIND_SAMPLEINST, TAUD_KIND_PATTERN, SAMPLEBIN_SIZE, INSTBIN_SIZE, SAMPLEINST_SIZE, SAMPLE_LEN_LIMIT, PATTERN_ROWS, PATTERN_BYTES, NUM_PATTERNS_MAX, NUM_CUES, CUE_SIZE, NUM_VOICES, NOTE_NOP, NOTE_KEYOFF, NOTE_FASTFADE, TAUD_C4, @@ -2382,10 +2397,15 @@ def build_pattern_bin(cells: dict, n_voices: int, return bytes(out) -def assemble_taud(sf: SF2, song: Song, layer_insts: list, meta_records: list, - slot_name: dict, pool: list, args) -> bytes: - speed, rpb = args.speed, args.rpb +def build_song_section(song: Song, speed: int, rpb: int, src_path: str, + args) -> dict: + """Per-song pattern/cue build shared by the full .taud and the .tpif paths. + Trims leading silence, emits the cell grid, plans cues, builds & dedupes the + pattern bin, and packs the cue sheet — everything that depends on this song's + notes but NOT on the (possibly shared) sample+instrument image. Returns a dict + carrying the compressed pattern bin / cue sheet plus the song-table and sMet + fields the container assemblers need.""" # Leading-silence trim: shift the grid so the first trigger is row 0. first_row = min(n.start_ft // speed for n in song.notes if n.slot > 0) shift_ft = first_row * speed @@ -2452,90 +2472,112 @@ def assemble_taud(sf: SF2, song: Song, layer_insts: list, meta_records: list, instr = CUE_INST_NOP sheet[ci*CUE_SIZE:(ci+1)*CUE_SIZE] = encode_cue(pats, instr) - # ── Sample + instrument bin ── - sampleinst_raw = build_sample_inst_bin(sf, pool, layer_insts, meta_records, - args.fadeout, bpm0, - force_synth_loop=args.force_synth_loop) - assert len(sampleinst_raw) == SAMPLEINST_SIZE - compressed = compress_blob(sampleinst_raw, "sample+inst bin") - comp_size = len(compressed) + # sMet beat divisions drive the tracker's row highlighting: primary = rows per + # NOTATED beat (the time-sig denominator), secondary = rows per bar. Using the + # denominator beat (not the rpb=rows-per-quarter) keeps the primary highlight a + # divisor of the bar — e.g. 7/8 → 2 rows (eighth), bar 14: 14 % 2 == 0, aligned; + # rpb=4 would drift (14 % 4 != 0). 4/4 → 4. + i = bisect.bisect_right(song.timesig_ft, shift_ft) - 1 + _, init_dpow = song.timesig[i] if i >= 0 else (4, 2) + beat_pri = max(1, round(rpb * 4 / (2 ** init_dpow))) + title = song.title or os.path.splitext(os.path.basename(src_path))[0] - pat_comp = compress_blob(pat_bin, "pattern bin") - cue_comp = compress_blob(bytes(sheet), "cue sheet") + return { + 'pat_comp': compress_blob(pat_bin, "pattern bin"), + 'cue_comp': compress_blob(bytes(sheet), "cue sheet"), + 'n_voices': n_voices, + 'n_unique': n_unique, + 'bpm0': bpm0, + 'speed': speed, + 'title': title, + 'beat_pri': max(1, min(255, beat_pri)), + 'beat_sec': max(1, min(255, init_bar_rows)), + } - song_table_off = TAUD_HEADER_SIZE + comp_size - song_off = song_table_off + TAUD_SONG_ENTRY - entry = encode_song_entry( + +def make_song_entry(section: dict, song_off: int, args) -> bytes: + """32-byte song-table row from a build_song_section() result.""" + return encode_song_entry( song_offset=song_off, - num_voices=n_voices, - num_patterns=n_unique, - bpm_stored=(bpm0 - 25) & 0xFF, - tick_rate=speed, + num_voices=section['n_voices'], + num_patterns=section['n_unique'], + bpm_stored=(section['bpm0'] - 25) & 0xFF, + tick_rate=section['speed'], base_note=0xA000, base_freq=8363.0, flags_byte=0x00, # linear pitch mode - pat_bin_comp_size=len(pat_comp), - cue_sheet_comp_size=len(cue_comp), + pat_bin_comp_size=len(section['pat_comp']), + cue_sheet_comp_size=len(section['cue_comp']), global_vol=0xFF, mixing_vol=args.mixing_vol, ) - # ── Project data: names + the Ixmp section recreating SF2 layering ── - proj_data = b'' - proj_off = 0 - if not args.no_project_data: - # Names indexed by slot (0 = unused). Layer slots carry the (suffixed) layer - # instrument name; meta slots carry the bare preset name. - max_slot = max([0] + list(slot_name)) - inst_names = ['' for _ in range(max_slot + 1)] - for s, nm in slot_name.items(): - inst_names[s] = nm - smp_names = [''] + [ms.name for ms in pool] - ixmp = {} - for ti in layer_insts: - if not ti.usable: - continue - pl = [p.to_ixmp_dict(ti.canonical, bpm0, args.fadeout) - for p in ti.patches if p is not ti.canonical] - if pl: - ixmp[ti.slot] = pl - if ixmp: - vprint(f" ixmp: {sum(len(p) for p in ixmp.values())} patch(es) " - f"across {len(ixmp)} instrument(s)") - title = song.title or os.path.splitext(os.path.basename(args.input))[0] - # sMet beat divisions drive the tracker's row highlighting: primary = - # rows per NOTATED beat (the time-sig denominator), secondary = rows per - # bar. Using the denominator beat (not the rpb=rows-per-quarter) keeps the - # primary highlight a divisor of the bar — e.g. 7/8 → 2 rows (eighth), bar - # 14: 14 % 2 == 0, aligned; rpb=4 would drift (14 % 4 != 0). 4/4 → 4. - i = bisect.bisect_right(song.timesig_ft, shift_ft) - 1 - _, init_dpow = song.timesig[i] if i >= 0 else (4, 2) - beat_pri = max(1, round(rpb * 4 / (2 ** init_dpow))) - song_meta = [{'index': 0, 'name': title, - 'notation': 240, # 24-TET (MIDI is 12-TET but 24 is harmless & cleaner pre-pitchbend transpose notation); 0 = raw/hex display - 'beat_pri': max(1, min(255, beat_pri)), - 'beat_sec': max(1, min(255, init_bar_rows))}] - proj_data = build_project_data( - project_name=title, - instrument_names=inst_names, - sample_names=smp_names, - song_metadata=song_meta, - ixmp_patches=ixmp or None, - ) +def make_song_meta(section: dict, index: int) -> dict: + """sMet entry (Project Data 's' block) from a build_song_section() result.""" + return {'index': index, 'name': section['title'], + 'notation': 240, # 24-TET (MIDI is 12-TET but 24 is harmless & cleaner pre-pitchbend transpose notation); 0 = raw/hex display + 'beat_pri': section['beat_pri'], + 'beat_sec': section['beat_sec']} + + +def build_sampleinst_blob(sf: SF2, pool: list, layer_insts: list, + meta_records: list, bpm0: int, args) -> bytes: + """Render + compress the sample+instrument image. MUST run before any + to_ixmp_dict() call, as it assigns each MonoSample's pool offset.""" + sampleinst_raw = build_sample_inst_bin(sf, pool, layer_insts, meta_records, + args.fadeout, bpm0, + force_synth_loop=args.force_synth_loop) + assert len(sampleinst_raw) == SAMPLEINST_SIZE + return compress_blob(sampleinst_raw, "sample+inst bin") + + +def build_inst_names(slot_name: dict, pool: list) -> tuple: + """(instrument_names, sample_names) slot-indexed lists for INam / SNam.""" + max_slot = max([0] + list(slot_name)) + inst_names = ['' for _ in range(max_slot + 1)] + for s, nm in slot_name.items(): + inst_names[s] = nm + smp_names = [''] + [ms.name for ms in pool] + return inst_names, smp_names + + +def build_ixmp(layer_insts: list, bpm0: int, args) -> dict: + """The Ixmp section recreating SF2 layering (instrument-id → patch dicts). + Reads each MonoSample's pool offset, so build_sampleinst_blob() must run first.""" + ixmp = {} + for ti in layer_insts: + if not ti.usable: + continue + pl = [p.to_ixmp_dict(ti.canonical, bpm0, args.fadeout) + for p in ti.patches if p is not ti.canonical] + if pl: + ixmp[ti.slot] = pl + if ixmp: + vprint(f" ixmp: {sum(len(p) for p in ixmp.values())} patch(es) " + f"across {len(ixmp)} instrument(s)") + return ixmp + + +def taud_header(kind: int, num_songs: int, comp_size: int) -> bytes: + """32-byte container header with projOff left at zero (patched by + finish_container when project data is present). `kind` is one of the + TAUD_KIND_* constants; the version byte is `kind | TAUD_VERSION`.""" header = (TAUD_MAGIC - + bytes([TAUD_VERSION, 1]) + + bytes([(kind & 0xC0) | TAUD_VERSION, num_songs & 0xFF]) + struct.pack(' bytes: + """Concatenate header + body parts + optional project data, patching projOff.""" + out = bytearray(taud_header(kind, num_songs, comp_size)) + for part in body_parts: + out += part if proj_data: proj_off = len(out) struct.pack_into(' bytes: + """Full single-song .taud file: sample+inst image + song table + patterns.""" + section = build_song_section(song, args.speed, args.rpb, args.input, args) + compressed = build_sampleinst_blob(sf, pool, layer_insts, meta_records, + section['bpm0'], args) + comp_size = len(compressed) + + song_off = TAUD_HEADER_SIZE + comp_size + TAUD_SONG_ENTRY + entry = make_song_entry(section, song_off, args) + + # ── Project data: names + Ixmp (I/S) + song metadata (s) + project name (P) ── + proj_data = b'' + if not args.no_project_data: + inst_names, smp_names = build_inst_names(slot_name, pool) + ixmp = build_ixmp(layer_insts, section['bpm0'], args) + proj_data = build_project_data( + project_name=section['title'], + instrument_names=inst_names, + sample_names=smp_names, + song_metadata=[make_song_meta(section, 0)], + ixmp_patches=ixmp or None, + ) + + return finish_container(TAUD_KIND_FULL, 1, comp_size, + [compressed, entry, section['pat_comp'], + section['cue_comp']], proj_data) + + +def assemble_tsii(sf: SF2, pool: list, layer_insts: list, meta_records: list, + slot_name: dict, bpm0: int, args) -> bytes: + """Sample and Instrument Image (.tsii): the shared sample+instrument bank for + a collection of .tpif pattern files (terranmon.txt:3342). numSongs = 0, no + song table / patterns; project data carries only the I/S blocks + (INam, SNam, Ixmp).""" + compressed = build_sampleinst_blob(sf, pool, layer_insts, meta_records, + bpm0, args) + proj_data = b'' + if not args.no_project_data: + inst_names, smp_names = build_inst_names(slot_name, pool) + ixmp = build_ixmp(layer_insts, bpm0, args) + proj_data = build_project_data( + instrument_names=inst_names, + sample_names=smp_names, + ixmp_patches=ixmp or None, + ) + return finish_container(TAUD_KIND_SAMPLEINST, 0, len(compressed), + [compressed], proj_data) + + +def assemble_tpif(sections: list, args) -> bytes: + """Pattern Image (.tpif): song table + patterns only, sharing the instruments + of a separately-loaded .tsii (terranmon.txt:3368). Sample+inst compSize = 0 + (section absent); project data carries only the p/s blocks (sMet here, no + pattern names). `sections` is a list of build_song_section() results — one + per song in the file.""" + n = len(sections) + table = bytearray() + blobs = bytearray() + # Pattern/cue data follows the whole song table; each entry points at its blob. + cursor = TAUD_HEADER_SIZE + n * TAUD_SONG_ENTRY + for sec in sections: + table += make_song_entry(sec, cursor, args) + blobs += sec['pat_comp'] + blobs += sec['cue_comp'] + cursor += len(sec['pat_comp']) + len(sec['cue_comp']) + + proj_data = b'' + if not args.no_project_data: + metas = [make_song_meta(sec, i) for i, sec in enumerate(sections)] + proj_data = build_project_data(song_metadata=metas) + + return finish_container(TAUD_KIND_PATTERN, n, 0, + [bytes(table), bytes(blobs)], proj_data) + + # ── Main ────────────────────────────────────────────────────────────────────── def main(): ap = argparse.ArgumentParser( description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter) - ap.add_argument('input', help='Input .mid file') + ap.add_argument('input', help='Input .mid file, OR a directory of MIDIs ' + '(batch mode → shared .tsii + per-file .tpif)') ap.add_argument('soundfont', help='SoundFont 2 (.sf2) sample library') ap.add_argument('output', nargs='?', default=None, - help='Output .taud (default: input stem + .taud)') + help='Output .taud (default: input stem + .taud). In directory ' + 'mode: output directory (default: the input directory)') ap.add_argument('--perc-force-mapping', nargs=2, type=int, default=None, metavar=('BANK', 'INST'), help='Force the percussion channel to this SF2 preset ' @@ -2615,37 +2735,52 @@ def main(): sys.exit("error: --max-layers must be 1..25") if not (0 <= args.mixing_vol <= 255): sys.exit("error: --mixingvol must be 0..255") - if args.output is None: - args.output = os.path.splitext(args.input)[0] + '.taud' - vprint(f"parsing MIDI '{args.input}'…") - division, merged = parse_midi(args.input) + if os.path.isdir(args.input): + run_directory(args) + else: + run_single(args) + + +# ── Pipeline helpers (shared by single-file and directory modes) ─────────────── + +def load_sf2_verbose(path: str) -> SF2: + vprint(f"parsing SF2 '{path}'…") + sf = parse_sf2(path) + vprint(f" {len(sf.presets)} preset(s), {len(sf.shdrs)} sample header(s)") + return sf + + +def load_midi_song(path: str, sf: SF2, args): + """Parse one MIDI into a Song with its resolved Taud grid, then apply SF2 + exclusive-class percussion choking. Returns (song, rpb, speed), or None when + the MIDI carries no playable notes.""" + vprint(f"parsing MIDI '{path}'…") + division, merged = parse_midi(path) # Resolve the Taud grid (Tickspeed + RPB) before mapping ticks to fine-ticks. # A pinned --rpb/--speed fixes that axis; the rest is auto-fit. - args.rpb, args.speed, timing_info = auto_timing( + rpb, speed, timing_info = auto_timing( division, merged, args.rpb, args.speed, args.max_voices) - vprint(f" timing: rpb {args.rpb}, speed {args.speed} ({timing_info})") + vprint(f" timing: rpb {rpb}, speed {speed} ({timing_info})") - song = extract_song(division, merged, args.rpb, args.speed) + song = extract_song(division, merged, rpb, speed) vprint(f" {len(song.notes)} note(s), {len(song.tempo_ft)} tempo event(s), " f"{len(song.timesig_ft)} time-signature event(s)") if not song.notes: - sys.exit("error: MIDI contains no playable notes") - - vprint(f"parsing SF2 '{args.soundfont}'…") - sf = parse_sf2(args.soundfont) - vprint(f" {len(sf.presets)} preset(s), {len(sf.shdrs)} sample header(s)") + return None # SF2 exclusiveClass percussion choking (closed hi-hat silences open hi-hat, etc.). apply_exclusive_class(song, sf, args.perc_force_mapping) + return song, rpb, speed - # Presets in first-use order; triggers keyed by the exact (noteVal-with-initial- - # bend, vol6) pair the patterns will carry, so layer trimming sees precisely what - # the engine matches at runtime. - slot_keys = [] - seen_keys = set() - triggers = {} + +def collect_triggers(song: Song, slot_keys: list, seen_keys: set, + triggers: dict) -> None: + """Append this song's presets (first-use order) to slot_keys and merge its + trigger (noteVal-with-initial-bend, vol6) histogram into `triggers`. The keys + match exactly what the patterns will carry, so layer trimming sees precisely + what the engine matches at runtime.""" for n in song.notes: if n.inst_key not in seen_keys: seen_keys.add(n.inst_key) @@ -2653,15 +2788,13 @@ def main(): t = triggers.setdefault(n.inst_key, {}) k = (key_to_noteval(n.key + n.bend0), round(n.vel * 63 / 127)) t[k] = t.get(k, 0) + 1 - vprint(f" {len(slot_keys)} preset(s) in use") - registry = {} - presets = build_presets(sf, slot_keys, triggers, args.perc_force_mapping, - registry, args.max_layers) - # Allocate instrument-bin slots: each layer is a normal instrument; a preset with - # >1 layer also takes a Metainstrument slot the note references. Single-layer - # presets stay plain instruments (no meta, no extra slot). +def allocate_slots(presets: dict, slot_keys: list): + """Assign instrument-bin slots across `slot_keys`. Each layer is a normal + instrument; a preset with >1 layer also takes a Metainstrument slot the note + references. Single-layer presets stay plain instruments (no meta, no extra + slot). Returns (layer_insts, meta_records, slot_name, note_slot).""" next_slot = 1 layer_insts = [] # all normal instruments, .slot assigned meta_records = [] # (meta_slot, name, [(layer_slot, bbox_rect)]) @@ -2691,8 +2824,12 @@ def main(): note_slot[ik] = meta_slot vprint(f" slots: {next_slot - 1} used — {len(layer_insts)} instrument(s), " f"{len(meta_records)} Metainstrument(s)") + return layer_insts, meta_records, slot_name, note_slot - # Tag notes with their trigger slot; notes whose preset failed to resolve drop. + +def tag_notes(song: Song, note_slot: dict) -> bool: + """Tag each note with its trigger slot and drop the unresolvable ones. Returns + True when the song keeps at least one note.""" unplayable = 0 for n in song.notes: n.slot = note_slot.get(n.inst_key, 0) @@ -2701,11 +2838,12 @@ def main(): if unplayable: vprint(f" warning: {unplayable} note(s) dropped (unresolvable preset)") song.notes = [n for n in song.notes if n.slot > 0] - if not song.notes: - sys.exit("error: no notes survived preset resolution") + return bool(song.notes) - # Pool = every sample referenced by a kept patch (canonical included), in - # deterministic first-reference order. Everything else is trimmed. + +def build_pool(layer_insts: list) -> list: + """Pool = every sample referenced by a kept patch (canonical included), in + deterministic first-reference order. Everything else is trimmed.""" pool = [] seen = set() for ti in layer_insts: @@ -2713,6 +2851,46 @@ def main(): if id(p.ms) not in seen: seen.add(id(p.ms)) pool.append(p.ms) + return pool + + +def find_midi_files(dir_path: str) -> list: + """Top-level .mid / .midi files in `dir_path`, sorted for deterministic order.""" + out = [] + for name in sorted(os.listdir(dir_path)): + full = os.path.join(dir_path, name) + if (os.path.isfile(full) + and os.path.splitext(name)[1].lower() in ('.mid', '.midi')): + out.append(full) + return out + + +# ── Conversion entry points ──────────────────────────────────────────────────── + +def run_single(args) -> None: + """Single MIDI → one self-contained .taud.""" + if args.output is None: + args.output = os.path.splitext(args.input)[0] + '.taud' + + sf = load_sf2_verbose(args.soundfont) + loaded = load_midi_song(args.input, sf, args) + if loaded is None: + sys.exit("error: MIDI contains no playable notes") + song, args.rpb, args.speed = loaded + + slot_keys, seen_keys, triggers = [], set(), {} + collect_triggers(song, slot_keys, seen_keys, triggers) + vprint(f" {len(slot_keys)} preset(s) in use") + + registry = {} + presets = build_presets(sf, slot_keys, triggers, args.perc_force_mapping, + registry, args.max_layers) + layer_insts, meta_records, slot_name, note_slot = allocate_slots( + presets, slot_keys) + + if not tag_notes(song, note_slot): + sys.exit("error: no notes survived preset resolution") + pool = build_pool(layer_insts) taud = assemble_taud(sf, song, layer_insts, meta_records, slot_name, pool, args) sf.file.close() @@ -2722,5 +2900,78 @@ def main(): print(f"wrote {len(taud)} bytes to '{args.output}'") +def run_directory(args) -> None: + """Directory of MIDIs → one shared .tsii (sample+instrument bank spanning the + union of every song) + one .tpif per MIDI (patterns only). terranmon.txt:3342.""" + out_dir = args.output or args.input + midis = find_midi_files(args.input) + if not midis: + sys.exit(f"error: no .mid/.midi files in directory '{args.input}'") + os.makedirs(out_dir, exist_ok=True) + vprint(f"directory mode: {len(midis)} MIDI file(s) → shared .tsii + per-file .tpif") + + sf = load_sf2_verbose(args.soundfont) + + # Phase 1: parse every MIDI, aggregating the preset/trigger universe so the + # shared instrument bank covers the union of all songs. + jobs = [] # (path, song, rpb, speed) for files with playable notes + slot_keys, seen_keys, triggers = [], set(), {} + for path in midis: + loaded = load_midi_song(path, sf, args) + if loaded is None: + vprint(f" warning: '{os.path.basename(path)}' has no playable notes — skipped") + continue + song, rpb, speed = loaded + collect_triggers(song, slot_keys, seen_keys, triggers) + jobs.append((path, song, rpb, speed)) + if not jobs: + sys.exit("error: no MIDI file produced playable notes") + vprint(f" {len(slot_keys)} preset(s) across {len(jobs)} song(s)") + + # Phase 2: build the one shared instrument set for the whole union. + registry = {} + presets = build_presets(sf, slot_keys, triggers, args.perc_force_mapping, + registry, args.max_layers) + layer_insts, meta_records, slot_name, note_slot = allocate_slots( + presets, slot_keys) + + # Phase 3: per song — tag notes against the shared slots, build the pattern + # section, and write its .tpif. (Independent of the sample+inst image below.) + sections = [] + for path, song, rpb, speed in jobs: + stem = os.path.splitext(os.path.basename(path))[0] + vprint(f"building '{stem}'…") + if not tag_notes(song, note_slot): + vprint(f" warning: '{stem}' lost all notes to preset resolution — skipped") + continue + section = build_song_section(song, speed, rpb, path, args) + tpif = assemble_tpif([section], args) + out_path = os.path.join(out_dir, stem + '.tpif') + with open(out_path, 'wb') as f: + f.write(tpif) + print(f"wrote {len(tpif)} bytes to '{out_path}'") + sections.append(section) + if not sections: + sys.exit("error: no song survived preset resolution") + + # Phase 4: the shared .tsii. Its fadeouts encode SF2 release times in seconds, + # but the engine fades per song-tick (rate ∝ BPM), so one image matches only one + # tempo exactly — target the mean of the songs' initial BPMs (override per-step + # with --fadeout). build_pool / build_sampleinst_blob run last because they + # assign the sample offsets the .tsii's Ixmp reads. + pool = build_pool(layer_insts) + ref_bpm0 = round(sum(s['bpm0'] for s in sections) / len(sections)) + vprint(f"building shared .tsii (reference BPM {ref_bpm0})…") + tsii = assemble_tsii(sf, pool, layer_insts, meta_records, slot_name, + ref_bpm0, args) + sf.file.close() + + sf_stem = os.path.splitext(os.path.basename(args.soundfont))[0] + tsii_path = os.path.join(out_dir, sf_stem + '.tsii') + with open(tsii_path, 'wb') as f: + f.write(tsii) + print(f"wrote {len(tsii)} bytes to '{tsii_path}'") + + if __name__ == '__main__': main() diff --git a/taud_common.py b/taud_common.py index 170245f..cd073c2 100644 --- a/taud_common.py +++ b/taud_common.py @@ -71,6 +71,18 @@ TAUD_MAGIC = bytes([0x1F,0x54,0x53,0x56,0x4D,0x61,0x75,0x64]) TAUD_VERSION = 1 TAUD_HEADER_SIZE = 32 # magic(8)+ver(1)+numSongs(1)+compSize(4)+projOff(4)+sig(14) TAUD_SONG_ENTRY = 32 # full spec entry (see encode_song_entry) + +# Container kind = top two bits of the version byte (terranmon.txt §.tsii / §.tpif, +# 3342-3401). The low six bits always carry TAUD_VERSION, so the full version byte +# is `kind | TAUD_VERSION`. +# 00 → full song file (.taud): sample+inst image + song table + patterns +# 10 → sample+instrument image (.tsii): numSongs = 0, no song table/patterns; +# project data carries only I*/S* blocks (INam, SNam, Ixmp) +# 11 → pattern image (.tpif): sample+inst compSize = 0 (section absent), song +# table + patterns present; project data carries only p*/s* blocks (pNam, sMet) +TAUD_KIND_FULL = 0x00 +TAUD_KIND_SAMPLEINST = 0x80 +TAUD_KIND_PATTERN = 0xC0 INST_RECORD_SIZE = 256 # widened 2026-05-06 (was 192). 256 inst × 256 = 64K. # Sample+instrument image (terranmon.txt:1985-1997, 2533-2564 — updated 2026-05-08). # Sample pool is now 8 MB, banked through MMIO 46 in 16 × 512 K windows. diff --git a/terranmon.txt b/terranmon.txt index 49da11b..06fbaeb 100644 --- a/terranmon.txt +++ b/terranmon.txt @@ -3339,6 +3339,68 @@ prefixes: -------------------------------------------------------------------------------- +**Taud Sample and Instrument Image Format (.tsii)** +Created by CuriousTorvald on 2026-06-15 + +This is a file format for storing sample+inst image only, intended for multiple Taud Pattern Images (.tpif) that share a same samples. + +Endianness: Little + +# Conformance language +Identical to **Taud serialisation format**. + +# File Structure + +\x1F T S V M a u d +[HEADER] +[SAMPLE+INSTRUMENT BIN IMAGE (GZip or Zstd compressed. Read 4-byte magic to determine)] +[PROJECT DATA] (optional) + [DATA BLOCKS WITH FOURCC HEADER (see Project Data section)] + +Basically a specialised "interpretation" of **Taud serialisation format**. The format differs in the following ways: + +* Empty song/pattern table. Number of songs is zero +* Version is always `0b10_xxxxxx`, where 'x' follows the expected Taud version +* PROJECT DATA has just enough blocks to represent instruments (ones starting with 'I' or 'S') + + + +**Taud Pattern Image Format (.tpif)** +Created by CuriousTorvald on 2026-06-15 + +This is a file format for storing patterns/songs only. + +Endianness: Little + +# Conformance language +Identical to **Taud serialisation format**. + +# File Structure + +\x1F T S V M a u d +[HEADER] +[SONG TABLE] +[PATTERN BIN for SONG 0 (GZip or Zstd compressed)] +[CUE SHEET for SONG 0 (GZip or Zstd compressed)] +[PATTERN BIN for SONG 1 (GZip or Zstd compressed)] +[CUE SHEET for SONG 1 (GZip or Zstd compressed)] +[PATTERN BIN for SONG 2 (GZip or Zstd compressed)] +[CUE SHEET for SONG 2 (GZip or Zstd compressed)] +... +[PROJECT DATA] (optional) + [DATA BLOCKS WITH FOURCC HEADER (see Project Data section)] + +Basically a specialised "interpretation" of **Taud serialisation format**. The format differs in the following ways: + +* Compressed size of SAMPLE+INST section is zero +* Version is always `0b11_xxxxxx`, where 'x' follows the expected Taud version +* PROJECT DATA has just enough blocks to represent patterns and songs (ones starting with 'p' or 's') + +# Intended Use Case +* Converting a collection of MIDI to Taud with single SoundFont for all. SoundFont is translated into .tsii, and MIDI files are translated into .tpif + +-------------------------------------------------------------------------------- + **S3M (ScreamTracker 3) to Taud conversion notes** (Implemented in s3m2taud.py) Created by CuriousTorvald on 2026-04-20