hi

how is everyone doing?

So im having a little issue right here... im trying to teach a friend of mine game development since months... ( we are in the same semester, bachelor of computer science )... he has good grades and is normally pretty smart, but he wont understand the easiest stuff... i gave him the exercise to code a small script that inserts/appends stuff to a textfield in a recursive way... e.g. first the amount, than the name using events/callbacks...

but he simply wont get it, how would you deal with such situations ?

Yea... hes doing his bachelor of computer science... fourth semester... he also has good grades :/

Thats why im despaired xD im literally trying to integrate him into my unity projects since over a year and he still only small understands parts of unity...

Its a wonder that i was able to teach him how a ecs is working... but i have no clue if its me... or if its him... either im a bad teacher, or he is just bad at it :/

For example references in unity... you simple declare a field and attach it via the inspector or via code... everytime he does this, he forgets it till the next time. Thats just a small example... or "logical cases"... for example "We should only insert something to a textfield when its set"... sometimes its like i would talk to someone never had a programming lesson before, but hes in the fourth semester xD nearly finished

@Tyyppi_77 Pretty much... but my code is fucking clean... patterns, modular, its honestly the best and easiest to understand stuff i have ever coded... back in the past i actually gave him dozends of tutorials to understand the mechanics e.g. once he created his own small project for some weeks, but he only played around a bit :/ i even wrote whole wiki pages for internal mechanics ( How does the database acess works... what does this system do and how is it structured... )

Well... thats probably true. I asked him several times, he always told me that hes interessted... but i somehow have the feeling that he is not that interessted at all :p which is a pitty because i dont find anyone else able to help me

Thats a good idea :) do you have any suggestions ?

@Tyyppi_77 Thanks ! Thats a good idea ^^ im gonna look at those

So my database issue is still persistent... Mainthread added a non persistent reference to a entity being updated -> Exception... I cant find any good solution for this, locking does not work because there no getter or setters... Snapshot does not work either ( tried to clone the components being saved )... How do others ecs solve this issues ? :/

I'm not sure other ECS use a database the way you're trying to. :/

Well i guess than its time to change the way i use the database ^^ but how ? Whats different ?

Can you walk us through how this error arises?

Thats probably a good idea... There 2 systems... One for scheduling saving jobs and one for scheduling updating jobs... Those jobs are getting executed multithreaded ( first saving, than updating ), thats pretty much were the database query gets constructed and executed...

Lets do a walk trough for the environment... Chunks are getting loaded, attach "Update" component, Update System construct the job with all entities marked as updated -> scheduled, Spawners getting either loaded or constructed and do spawn resources ( all marked for being saved ), chunk assignment system iterates over entities to put them into a chunk (modified updating entity), update job executes and causes a exception because the state of the updating entity was modifed

Why does the update job care about your database?

How do you mean that ? Because thats the purpose of it ^^ executing the update query in another thread for preventing a mainthread block

The load job can handle reading from your database, the save job can handle writing to your database. The update job should just be manipulating data in RAM.

Thats what the save job already does ^^ it persists entities not being saved... The update job on the other site triggers a "update query"... Or do we mean different "updates" ?

It sounds like you are using this word "update" in a very different way than games conventionally do.

Thats true... In my case i mean writing entity updates to the database ^^

That sounds a lot like "saving"

Which is different from saving entities... Different query syntax... "insert into"/"update x from y where..."

But yeah basically its both saving ^^

Ah, you're differentiating between "save newly-created items" and "save changes to existing items"

Exactly :D

Save newly = Save System, Job / Save existing = Update System, Job

The issue occurs when we save a existing chunk but it has been modified by the mainthread and references a new unsaved resource

Ie. your two database-writing jobs have to occur sequentially, without the potential to create new items in-between them.

Thanks ! But what if theres already a "Update" job running and new items getting created, saved and the this way the entities being updated are getting modified again ? :/ My jobs are already running sequential... But the chunk update is fired before any new resources are constructed... Then we create new resources, create a saving job and schedule it :/ same problem

This approach works beautifull on a single thread... But there need to be a way to realize this inside a multithreaded environment

If you need to multithread saving, it's done with either locks or a snapshot. You fundamentally cannot safely save data that you might still be modifying.

Thanks :) locking... Thats something i havent looked into yet... Not that beautifull, but if it works, it works. What about snapshots ? Could we simply clone the entities with their components, insert them to a seperate world and than save those cloned entities ? Or how does that snapshot stuff works ? ^^ i guess we need to create those snapshots on the mainthread ?

You can use a form of double-buffering for this. Keep two copies of your game state. On odd frames you read from copy 0 and write to copy 1, on even frames you read from copy 1 and write to copy 0. Then your saving routine has a read-only copy to read from while the rest of your game is updating. You just need all jobs to complete before you flip.

Alright, this sounds interessting... How do we do this ? Simply deep cloning them ? And doesnt this come along a major lag every when we copy the whole game state every second frame ?

The joy of this is you never have to do a full copy at all. The act of writing the updated state of your entities is what creates the new copy.

You can effectively nuke the writable buffer at the start of each frame, and create it from scratch as you write out the updated versions of each piece of data.

Hmmm... Thats something i dont fully understand... Could you please clarify this ? ^^

newGameState.position could have been completely empty before this moment. The act of updating the state populates the state.

You just need to make sure every piece of meaningful state data gets a chance to be pushed through like that. Or is in a separate read-only collection that can be shared, if it's not something that can change frame to frame.

This can also be good for other multithreading in your game. Now even jobs that read data A can run at the same time as jobs that write data A, because the readable and writable versions are separate. So, you have much less work relating to scheduling dependencies, locks, etc.

The cost is that it can take longer for a change to propagate through multiple passes of updates. But you still have the power to read from the writable data if you know the last set of writes have finished and need to propagate the latest changes faster.

Ah i understand, but this does not work in every case, right ? Lets say we have a chunk component containing a list of references to positions laying inside the chunk... Furthermore i have no clue how to realize this in the current ecs approach, it looks like tons of work... Probably i would need to change the internal workflow ( which i have no acess to ) :o

Your "references" here aren't pointers to specific memory locations, but IDs/indexes that are portable. So I can look up the corresponding item in either the old or new data sets.

And what happens if we modify a component at multiple locations ? Both times we would change the new state, but the oldstate is the same which results in a override

Right, but if you're writing the same component from multiple jobs that could run concurrently in your current system, that's also unsafe.

So if you have multiple writes, you need them to occur in a well-defined order, with the later one honouring the output of the earlier one.

So that case still needs scheduling/locking, in both versions.

I somehow get the feeling that a ecs is either incredible fast but pretty unflexible in terms of systems and their components or pretty flexible but slower... As nice as such a double buffered system sounds, i think im not able to integrate it into "artemis odb - java" :/ so we are back at the beginning... Either locks or deep cloning the entities being saved