async loading (hopefully?)

This commit is contained in:
minjaesong
2026-04-04 17:05:00 +09:00
parent 981418d0c7
commit 6118f30d5f
6 changed files with 344 additions and 123 deletions

23
.idea/csv-editor.xml generated Normal file
View File

@@ -0,0 +1,23 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="CsvFileAttributes">
<option name="attributeMap">
<map>
<entry key="/Terrarum.wiki/Items.md">
<value>
<Attribute>
<option name="separator" value="," />
</Attribute>
</value>
</entry>
<entry key="/Terrarum.wiki/Modules:Setup.md">
<value>
<Attribute>
<option name="separator" value="," />
</Attribute>
</value>
</entry>
</map>
</option>
</component>
</project>

176
CLAUDE.md Normal file
View File

@@ -0,0 +1,176 @@
# Terrarum
A modular 2D side-scrolling tilemap platformer engine and game, built on LibGDX with Kotlin/Java. GPL-3.0.
## Build & Run
- **IDE**: IntelliJ IDEA (project files: `Terrarum_renewed.iml`, `TerrarumBuild.iml`)
- **JDK**: 21
- **Language**: Kotlin + Java mixed; Kotlin is primary, `App.java` and `FrameBufferManager.java` are Java
- **Dependencies**: All libraries are in `lib/` (fat-jar approach, no Maven/Gradle). Includes LibGDX, GraalVM JS (modified), dyn4j (Vector2 only), custom bitmap font lib
- **Entry point**: `net.torvald.terrarum.Principii.main()` (Java) which launches `App` (the LibGDX `ApplicationListener`)
- **Distribution**: `buildapp/` scripts produce per-platform bundles with jlink'd runtimes
- **Assets**: `assets/` for development, `assets_release/` for distribution. Custom `.tevd` archive format for release assets (`AssetCache.kt`)
## Project Structure
```
src/net/torvald/terrarum/
App.java -- Main application class (LibGDX ApplicationListener). GL thread, render loop, splash screen
CommonResourcePool.kt -- Thread-safe GL resource loading with dispatch queues
ModMgr.kt -- Module manager. Scans/loads game modules, provides getGdxFile/getJavaClass
IngameInstance.kt -- Base class for game screens (show/hide/render/resize lifecycle)
Terrarum.kt -- Game-level singleton (ingame instance management, extension functions)
GameUpdateGovernor.kt -- Update/render tick governors (ConsistentUpdateRate, Anarchy)
MusicService.kt -- Music playback management
modulebasegame/
EntryPoint.kt -- Basegame module entry point (registers items, fixtures, blocks, weather)
TitleScreen.kt -- Title screen (demo world rendering, async loading)
TerrarumIngame.kt -- Main gameplay screen
IngameRenderer.kt -- World rendering pipeline (FBO composition, lightmap, blur, shadows)
BuildingMaker.kt -- Building editor screen
worlddrawer/
WorldCamera.kt -- Camera position tracking (follows player, interpolated movement)
LightmapRenderer.kt -- Per-tile RGB+UV light calculation and rasterisation
BlocksDrawer.kt -- Tile rendering
FeaturesDrawer.kt -- Wire/feature overlay rendering
gamecontroller/
IME.kt -- Input Method Engine (GraalVM JS keyboard layouts, loaded on daemon thread)
weather/
WeatherMixer.kt -- Weather state machine, skybox rendering
SkyboxModelHosek.kt -- Physically-based sky model
```
## Architecture: Threading & GL Dispatch
All OpenGL calls (Texture, ShaderProgram, FrameBuffer creation) **must** happen on the GL thread. The codebase uses a dispatch mechanism to safely create GL resources from background threads.
### CommonResourcePool (the dispatch hub)
```
Background Thread GL Thread (App.render)
| |
|-- addToLoadingList("name", { Texture() })|
|-- loadAll() |
| | |
| +-- if on GL thread: run directly |
| +-- if not: enqueue to |
| glDispatchQueue + latch.await() |
| CommonResourcePool.update()
| |-- poll glDispatchQueue
| | run loadfun, latch.countDown()
| |-- poll glRunnableQueue
| | run block, latch.countDown()
| |-- poll slowLoadingQueue (one per tick)
| |
+-- latch released, continues <------------+
```
Key methods:
- `loadAll()` -- batch load; blocks background thread until GL thread processes
- `loadAllSlowly()` -- timesliced; one resource per tick via `update()`
- `runOnGLThread { }` -- run arbitrary block on GL thread, blocking caller
- `update()` -- called every tick from `App.render()`, processes all queues
- `getOrPut(name, loadfun, killfun)` -- thread-safe get-or-create with GL dispatch
### App.java Boot Sequence
```
create()
-> postInit() [GL thread, synchronous]
|-- load shaders, fonts, audio device, IME
|-- CommonResourcePool.setGLThread(currentThread)
|-- spawn Terrarum-PostInitLoader thread:
| ModMgr.invoke() // module entry points (dispatched to GL via runOnGLThread)
| CommonResourcePool.loadAllSlowly() // timesliced resource loading
| loadingThreadDone = true
+-- return (non-blocking)
render() loop [GL thread]
while currentScreen == null:
|-- drawSplash()
|-- CommonResourcePool.update() // process GL dispatch queues
|-- when loadingThreadDone && loaded:
| postLoadInit() // tile atlas, audio mixer, Terrarum.initialise()
| setScreen(titleScreen) // transitions to title screen
```
### TitleScreen Async Loading
`TitleScreen.show()` returns immediately after starting a background thread. The splash screen continues displaying until all loading completes.
```
show()
|-- quick GL setup (viewport, input processor, FBO, savegame list)
|-- spawn Terrarum-TitleScreenLoader thread:
| load demo world, compute camera nodes, bogoflops, audio reset
| backgroundLoadDone = true
+-- return
renderImpl() [GL thread, every frame]
if !loadDone:
|-- App.drawSplash()
|-- processGLLoadStep() // state machine, one step per frame:
| 0: wait for backgroundLoadDone
| 1: SkyboxModelHosek.loadlut()
| 2: IngameRenderer.setRenderedWorld + WeatherMixer init
| 3: load halfgrad texture
| 4: UIFakeGradOverlay
| 5: UIFakeBlurOverlay
| 6: UIRemoCon
| 7: MusicService.enterScene, gameUpdateGovernor.reset(), loadDone=true
else:
normal title screen rendering (world + UI via IngameRenderer)
```
### IME Loading
`IME.kt` object `init {}` spawns a `Terrarum-IMELoader` daemon thread for GraalVM JS context binding and key layout/IME file scanning. Icon texture loading remains synchronous (requires GL thread).
### ModMgr Class Initialisation
`ModMgr.kt` object `init {}` only does metadata loading and class instantiation (fast). The heavy `invoke()` method runs entry points wrapped in `CommonResourcePool.runOnGLThread { }` to avoid GL calls on the wrong thread. This separation prevents `<clinit>` lock deadlocks where a background thread holding the class init lock blocks on a GL dispatch latch while the GL thread tries to access the same class.
## Architecture: Rendering Pipeline
### IngameRenderer
Singleton object. Lazily initialised on first `invoke()` call via `invokeInit()`.
Render path (per frame):
1. `LightmapRenderer.recalculate()` -- every 3 frames, computes per-tile RGBA light values
2. `prepLightmapRGBA()` -- Kawase blur on lightmap into `lightmapFbo`
3. `BlocksDrawer.renderData()` -- prepare tile draw data
4. `drawToRGB()` -- render world tiles + actors into `fboRGB` (with shadow FBOs)
5. `drawToA()` -- render alpha/glow channel
6. Composite: sky -> world -> light multiply -> emissive blend -> vibrancy -> UI
7. Output to `App.renderFBO`, then `TerrarumPostProcessor.draw()` applies final effects
**Critical ordering in `resize()`**: `BlocksDrawer.resize()` and `LightmapRenderer.resize()` must be called **before** `lightmapFbo` creation, because `lightmapFbo` dimensions derive from `LightmapRenderer.lightBuffer` size.
### WorldCamera
Follows an `ActorWithBody` (player or camera actor). Position interpolated per frame. `WorldCamera.x/y` = top-left corner of visible area. `gdxCamX/Y` = centre of visible area. `moveCameraToWorldCoord()` positions the IngameRenderer camera for world-space drawing; `setCameraPosition(0,0)` positions it for screen-space drawing.
### FBO Extension Functions
- `FrameBuffer.inAction(camera, batch) { }` -- bind FBO, set camera to FBO dimensions (Y-down), run block, restore camera to screen dimensions
- `FrameBuffer.inActionF(camera, batch) { }` -- same but Y-up (for flipped FBO content)
Both save/restore `camera.position` and call `setToOrtho` with `App.scr.wf/hf` on exit.
## Key Conventions
- **GL thread safety**: Any code creating `Texture`, `TextureRegion`, `TextureRegionPack`, `ShaderProgram`, `FrameBuffer`, or `Pixmap` must run on the GL thread. Use `CommonResourcePool.runOnGLThread { }` when on a background thread.
- **Kotlin `object` singletons**: First access triggers `<clinit>`. Keep `init {}` blocks fast (no blocking, no GL dispatch). Defer heavy work to explicit `invoke()` methods.
- **`ConcurrentHashMap` cannot store null values**. Use `HashMap` for maps that need nullable values (e.g. `poolKillFun`).
- **`@Volatile`** for cross-thread boolean flags (`loadDone`, `backgroundLoadDone`, etc.)
- **`lateinit var` guards**: Use `::prop.isInitialized` checks in `resize()` and `dispose()` for properties set during async loading steps.
- **`ConsistentUpdateRate`**: Accumulates delta time; may call `updateFunction` multiple times before `renderFunction`. Call `.reset()` before transitioning to active rendering to avoid catch-up storms.
- **Textures**: Use TGA format. Images with semitransparency must be TGA; opaque images may be PNG.
- **Coordinate system**: Y-down (camera `setToOrtho(true, ...)`). `setCameraPosition(0, 0)` places origin at screen top-left.
- **ROUNDWORLD**: World wraps horizontally. `WorldCamera.x` uses `fmod worldWidth`. Lightmap and tile drawing account for wrapping.

Binary file not shown.

View File

@@ -865,7 +865,7 @@ public class App implements ApplicationListener {
return ditherPatterns[hash % ditherPatterns.length];
}
private void drawSplash() {
public static void drawSplash() {
setCameraPosition(0f, 0f);
logoBatch.setColor(Color.WHITE);
@@ -874,7 +874,7 @@ public class App implements ApplicationListener {
int drawWidth = Toolkit.INSTANCE.getDrawWidth();
int safetyTextLen = fontGame.getWidth(Lang.INSTANCE.get("APP_WARNING_HEALTH_AND_SAFETY", true));
int logoPosX = (drawWidth - splashScreenLogo.getRegionWidth() - safetyTextLen) >>> 1;
int logoPosX0 = (drawWidth - splashScreenLogo.getRegionWidth() - safetyTextLen) >>> 1;
int logoPosY = Math.round(scr.getHeight() / 15f);
int textY = logoPosY + splashScreenLogo.getRegionHeight() - 16;
@@ -900,7 +900,7 @@ public class App implements ApplicationListener {
logoBatch.end();
}
int logoPosX = (int)(logoPosX0 + Math.round(100 * Math.sin(GLOBAL_RENDER_TIMER / 50.0)));
// draw logo reflection
logoBatch.setShader(shaderReflect);
@@ -1448,7 +1448,7 @@ public class App implements ApplicationListener {
}
private void setCameraPosition(float newX, float newY) {
private static void setCameraPosition(float newX, float newY) {
camera.position.set((-newX + scr.getWidth() / 2), (-newY + scr.getHeight() / 2), 0f); // deliberate integer division
camera.update();
logoBatch.setProjectionMatrix(camera.combined);

View File

@@ -1307,6 +1307,11 @@ object IngameRenderer : Disposable {
//fboBlurQuarter.dispose()
}
// BlocksDrawer and LightmapRenderer must be resized before lightmapFbo is created,
// because lightmapFbo dimensions are derived from LightmapRenderer.lightBuffer.
BlocksDrawer.resize(width, height)
LightmapRenderer.resize(width, height)
fboRGB = Float16FrameBuffer(width, height, false)
fboRGB_lightMixed0 = Float16FrameBuffer(width, height, false)
fboRGB_lightMixed = Float16FrameBuffer(width, height, false)
@@ -1345,9 +1350,6 @@ object IngameRenderer : Disposable {
false
)*/
BlocksDrawer.resize(width, height)
LightmapRenderer.resize(width, height)
blurWriteQuad.setVertices(floatArrayOf(
0f,0f,0f, 1f,1f,1f,1f, 0f,1f,

View File

@@ -66,7 +66,10 @@ class TitleScreen(batch: FlippingSpriteBatch) : IngameInstance(batch) {
}
//private var loadDone = false // not required; draw-while-loading is implemented in the AppLoader
private var loadDone = false
@Volatile private var backgroundLoadDone = false
private var glLoadStep = 0
private lateinit var titleLoadingThread: Thread
private lateinit var demoWorld: GameWorld
private lateinit var cameraNodes: FloatArray // camera Y-pos
@@ -156,117 +159,129 @@ class TitleScreen(batch: FlippingSpriteBatch) : IngameInstance(batch) {
gameUpdateGovernor = ConsistentUpdateRate.also { it.reset() }
}
private fun loadThingsWhileIntroIsVisible() {
printdbg(this, "Intro pre-load")
private fun startBackgroundLoading() {
titleLoadingThread = Thread({
printdbg(this, "Background loading started")
try {
val fileHandle = ModMgr.getGdxFile("basegame", "demoworld")
val reader = fileHandle.reader("UTF-8")
//ReadWorld.readWorldAndSetNewWorld(Terrarum.ingame!! as TerrarumIngame, reader)
val world = ReadSimpleWorld(reader, fileHandle.file())
demoWorld = world
demoWorld.worldTime.timeDelta = 30
printdbg(this, "Demo world loaded")
}
catch (e: IOException) {
demoWorld = GameWorld(LandUtil.CHUNK_W, LandUtil.CHUNK_H, 0L, 0L)
demoWorld.worldTime.timeDelta = 30
printdbg(this, "Demo world not found, using empty world")
}
demoWorld.renumberTilesAfterLoad()
this.world = demoWorld
// set initial time to summer
demoWorld.worldTime.addTime(WorldTime.DAY_LENGTH * 32)
// construct camera nodes
val nodeCount = demoWorld.width / cameraNodeWidth
cameraNodes = kotlin.FloatArray(nodeCount) {
val tileXPos = (demoWorld.width.toFloat() * it / nodeCount).floorToInt()
var travelDownCounter = 0
while (travelDownCounter < demoWorld.height &&
!BlockCodex[demoWorld.getTileFromTerrain(tileXPos, travelDownCounter)].isSolid
// !BlockCodex[demoWorld.getTileFromWall(tileXPos, travelDownCounter)].isSolid
) {
travelDownCounter += 2
try {
val fileHandle = ModMgr.getGdxFile("basegame", "demoworld")
val reader = fileHandle.reader("UTF-8")
val world = ReadSimpleWorld(reader, fileHandle.file())
demoWorld = world
demoWorld.worldTime.timeDelta = 30
printdbg(this, "Demo world loaded")
}
catch (e: IOException) {
demoWorld = GameWorld(LandUtil.CHUNK_W, LandUtil.CHUNK_H, 0L, 0L)
demoWorld.worldTime.timeDelta = 30
printdbg(this, "Demo world not found, using empty world")
}
travelDownCounter * TILE_SIZEF
}
// apply gaussian blur to the camera nodes
for (i in cameraNodes.indices) {
// val offM2 = cameraNodes[(i-2) fmod cameraNodes.size] * 1f
val offM1 = cameraNodes[(i-1) fmod cameraNodes.size] * 1f
val off0 = cameraNodes[i] * 2f
val off1 = cameraNodes[(i+1) fmod cameraNodes.size] * 1f
// val off2 = cameraNodes[(i+2) fmod cameraNodes.size] * 1f
// cameraNodes[i] = (offM2 + offM1 + off0 + off1 + off2) / 16f
cameraNodes[i] = (offM1 + off0 + off1) / 4f
}
demoWorld.renumberTilesAfterLoad()
this.world = demoWorld
// set initial time to summer
demoWorld.worldTime.addTime(WorldTime.DAY_LENGTH * 32)
// construct camera nodes
val nodeCount = demoWorld.width / cameraNodeWidth
cameraNodes = kotlin.FloatArray(nodeCount) {
val tileXPos = (demoWorld.width.toFloat() * it / nodeCount).floorToInt()
var travelDownCounter = 0
while (travelDownCounter < demoWorld.height &&
!BlockCodex[demoWorld.getTileFromTerrain(tileXPos, travelDownCounter)].isSolid
) {
travelDownCounter += 2
}
travelDownCounter * TILE_SIZEF
}
// apply gaussian blur to the camera nodes
for (i in cameraNodes.indices) {
val offM1 = cameraNodes[(i-1) fmod cameraNodes.size] * 1f
val off0 = cameraNodes[i] * 2f
val off1 = cameraNodes[(i+1) fmod cameraNodes.size] * 1f
cameraNodes[i] = (offM1 + off0 + off1) / 4f
}
cameraPlayer = CameraPlayer(demoWorld, cameraAI)
cameraPlayer = CameraPlayer(demoWorld, cameraAI)
CommandDict // invoke
IngameRenderer.setRenderedWorld(demoWorld)
WeatherMixer.internalReset(this)
WeatherMixer.titleScreenInitWeather(demoWorld.weatherbox)
WeatherMixer.forceTimeAt = 23456
// measure bogoflops here
val bogoflopsOld = App.bogoflops
App.updateBogoflops(100_000_000L)
printdbg(this, "Bogoflops old: $bogoflopsOld new: ${App.bogoflops}")
App.bogoflops = maxOf(App.bogoflops, bogoflopsOld)
App.audioMixer.ambientTracks.forEach {
it.stop()
it.currentTrack = null
it.nextTrack = null
}
App.audioMixer.reset()
// load a half-gradient texture that would be used throughout the titlescreen and its sub UIs
CommonResourcePool.addToLoadingList("title_halfgrad") {
Texture(AssetCache.getFileHandle("graphics/halfgrad.tga")).also {
it.setFilter(Texture.TextureFilter.Linear, Texture.TextureFilter.Linear)
printdbg(this, "Background loading done")
backgroundLoadDone = true
}, "Terrarum-TitleScreenLoader")
titleLoadingThread.isDaemon = true
titleLoadingThread.start()
}
private fun processGLLoadStep() {
when (glLoadStep) {
0 -> {
// Wait for background thread to finish world loading + CPU work
if (backgroundLoadDone) glLoadStep++
}
1 -> {
SkyboxModelHosek.loadlut()
glLoadStep++
}
2 -> {
IngameRenderer.setRenderedWorld(demoWorld)
WeatherMixer.internalReset(this)
WeatherMixer.titleScreenInitWeather(demoWorld.weatherbox)
WeatherMixer.forceTimeAt = 23456
glLoadStep++
}
3 -> {
// Load half-gradient texture
CommonResourcePool.addToLoadingList("title_halfgrad") {
Texture(AssetCache.getFileHandle("graphics/halfgrad.tga")).also {
it.setFilter(Texture.TextureFilter.Linear, Texture.TextureFilter.Linear)
}
}
CommonResourcePool.loadAll()
glLoadStep++
}
4 -> {
// Gradient overlay UI
val uiFakeGradOverlay = UIFakeGradOverlay()
uiFakeGradOverlay.setPosition(0, 0)
uiContainer.add(uiFakeGradOverlay)
glLoadStep++
}
5 -> {
// Blur overlay UI
uiFakeBlurOverlay = UIFakeBlurOverlay(1f, false)
uiFakeBlurOverlay.setPosition(0, 0)
uiContainer.add(uiFakeBlurOverlay)
glLoadStep++
}
6 -> {
// Remote control UI
uiRemoCon = UIRemoCon(this, UITitleRemoConYaml(App.savegamePlayers.isNotEmpty()))
uiRemoCon.setPosition(0, 0)
uiRemoCon.setAsOpen()
uiContainer.add(uiRemoCon)
glLoadStep++
}
7 -> {
MusicService.enterScene("title")
gameUpdateGovernor.reset()
loadDone = true
}
}
CommonResourcePool.loadAll()
// fake UI for gradient overlay
val uiFakeGradOverlay = UIFakeGradOverlay()
uiFakeGradOverlay.setPosition(0, 0)
uiContainer.add(uiFakeGradOverlay)
// fake UI for blur
uiFakeBlurOverlay = UIFakeBlurOverlay(1f, false)
uiFakeBlurOverlay.setPosition(0,0)
uiContainer.add(uiFakeBlurOverlay)
uiRemoCon = UIRemoCon(this, UITitleRemoConYaml(App.savegamePlayers.isNotEmpty()))
uiRemoCon.setPosition(0, 0)
uiRemoCon.setAsOpen()
uiContainer.add(uiRemoCon)
CommandDict // invoke
SkyboxModelHosek.loadlut() // invoke
// Skybox.initiate() // invoke the lengthy calculation
// TODO add console here
//loadDone = true
// measure bogoflops here
val bogoflopsOld = App.bogoflops
App.updateBogoflops(100_000_000L)
printdbg(this, "Bogoflops old: $bogoflopsOld new: ${App.bogoflops}")
App.bogoflops = maxOf(App.bogoflops, bogoflopsOld)
App.audioMixer.ambientTracks.forEach {
it.stop()
it.currentTrack = null
it.nextTrack = null
}
App.audioMixer.reset()
}
@@ -294,10 +309,8 @@ class TitleScreen(batch: FlippingSpriteBatch) : IngameInstance(batch) {
App.updateListOfSavegames()
UILoadGovernor.reset()
loadThingsWhileIntroIsVisible()
startBackgroundLoading()
printdbg(this, "show() exit")
MusicService.enterScene("title")
}
@@ -305,6 +318,12 @@ class TitleScreen(batch: FlippingSpriteBatch) : IngameInstance(batch) {
private var introUncoverDeltaCounter = 0f
override fun renderImpl(updateRate: Float) {
if (!loadDone) {
App.drawSplash()
processGLLoadStep()
return
}
IngameRenderer.setRenderedWorld(demoWorld)
super.renderImpl(updateRate)
@@ -549,23 +568,24 @@ class TitleScreen(batch: FlippingSpriteBatch) : IngameInstance(batch) {
initViewPort(App.scr.width, App.scr.height)
// resize UI by re-creating it (!!)
uiRemoCon.resize(App.scr.width, App.scr.height)
// TODO I forgot what the fuck kind of hack I was talking about
//uiMenu.setPosition(0, UITitleRemoConRoot.menubarOffY)
uiRemoCon.setPosition(0, 0) // shitty hack. Could be:
// 1: Init code and resize code are different
// 2: The UI is coded shit
if (::uiRemoCon.isInitialized) {
// resize UI by re-creating it (!!)
uiRemoCon.resize(App.scr.width, App.scr.height)
// TODO I forgot what the fuck kind of hack I was talking about
//uiMenu.setPosition(0, UITitleRemoConRoot.menubarOffY)
uiRemoCon.setPosition(0, 0) // shitty hack. Could be:
// 1: Init code and resize code are different
// 2: The UI is coded shit
IngameRenderer.resize(App.scr.width, App.scr.height)
IngameRenderer.resize(App.scr.width, App.scr.height)
}
printdbg(this, "resize() exit")
}
override fun dispose() {
uiRemoCon.dispose()
demoWorld.dispose()
if (::uiRemoCon.isInitialized) uiRemoCon.dispose()
if (::demoWorld.isInitialized) demoWorld.dispose()
warning32bitJavaIcon.texture.dispose()
warningAppleRosettaIcon.texture.dispose()
}