I think most path-following in games I"ve worked on has been done with Bezier splines. I haven't seen any subdivision to straight segments except for rendering or collisions, though it could be it's hidden in bits of the code I haven't been in.
Okay, thanks; I need to do more research because I wonder how an agent would actually follow this curve. AFAIK, a param t is used to determine where one is on the curve, but for each equal subdivision of t, the segment of the curve is not equivalent.
if the car has a few sideway sensors, it could as well learn it is in a curve (if they are X shaped, that should be enough for a single path track, regardless of curves)
there was as well one video from CodeMonkey that trained the Unity mlagents for racing cars
Ah, yes, I don't have access to these tools; I'll take a look because this may inspire future work.
For now the work I'm doing is having a "robot" forklift driving around in a warehouse, randomly.
"randomly" -> randomly select an intersection, go there, rinse, repeat.
@Zibelas "if the car has a few sideway sensors, it could as well learn it is in a curve" so the car detects the sides of the track and drive accordingly?
Okay; maybe we'll do it like this for the next iteration; for now I'm planning to have the robot just "chase a runner" that is on a line; and if something is on the line, stop and wait.
if the agent should learn all by itself, just reward it the longer it can stay on the track and give it a penalty when hitting the wall. It will learn to average the sensors to roughtly the middle of the track
the great thing is, if set up correctly, after training it once (that means like a few 10k iterations), you can change the track into what ever and it will still drive
personally I really enjoyed working with the ml agents, the hardest thing for me was to figure out how to use python in the console to start the training
the actual coding of the agents was trivial in comparison, but writing the trainingsparameter took some effort
my best success was using it as an ai in a card board game. All the agent got as input were all valid moves it could take per round, from there it had to figure out how to win the game
in that game, I used as a metric how many rounds it takes to finish the game. The absolut perfect game (if the right random cards are open, nobody destroys your game, etc) is like 13 rounds. 20 is my usual turn. I got some ai model with average on 19 and another one at 23
since the decisions are always statebased (or at least in that game), you could exchange the model on the fly
but even with the same model, if you give it some variance, it will give results like: 80% A and 20% B for which option it should take
the agent will usually pick A, but you have some kind of curiosity factor that sometimes it takes the worse option
to figure out if not down the path there is some better result
the longer you train the agent and how you tweek its curiosity over the trainings cycle, the more or less it will play sometimes either brilliant or just plain stupid
the rules were really simple for playing - take 3 different ressource - take 2 of the same ressource - buy a card with ressources (cards are giving permanent ressource of one type)
so all the agent had to do was decide between which of the 3 options it should take. But there are 5 different ressources and always 12 cards open for buying
the agent had to learn it needs to aim for one card and pick the ressources accordingly
after that it had to aim for getting as many cards as fast as possible (or at least with the most victory points)
the agent is not really portable, since it needs to train for each game. But I think you can gain some other results that you usually dont get yourself
for example different winning strategies
I would say if there is only one valid strategy, the game gets fast boring
but if the ai finds 10 different strategies to win AND each one can win against a different one, that is a big plus
(about using the agent to optimize gameplay for other games, I meant, coding a different agent) I remember a turn based web browser text game I played back in the day, and the rules were quite standard, so one could figure the best way to achieve something like "maximize land expansion" or "maximize defence" or something..
it always also depends on how much free choice you give the agent
for example if you say prio "maximize land expansion" that might work but there could be a simple counter strategy with "Build raiders"
the land ai never builds defense since it got more rewards when it picked the choice for land expansion
but if you have a clear winning condition (like other player is defeated, reached X points, have army of 100, have 25 lands, etc)
and just reward your agent at the end of the game with: you did good, you did bad
or your move was a setback, you lost 25% of your reward you earned so far
let it play 100.000 times against itself
and in the end it will have a valid strategy
after that you train a second new ai and let it play against your previous ai.
i think training around 100k games took around 10min; there are options for running multiple games in parallel and since it just needs input/ output, all animations/ grafics can be skipped
in the end you have an Ai that might not have the strongest single best strategy but is really good at knowing not to loose. you can as well pick specialist AI from the results (like aggressive/ defensive)
and if the ai is too good/ bad, you can always still adjust the odds over the game itself (like ai gets 25% more ressources at the same time or draws better cards with a higher chance)
in the end the goal is to have an enjoyable opponent
I mean if I don't know a certain strategy, it can't really code my opponent to use it
actually no
I wrote first the game fully playable and code the AI afterwards
a realistic AI agent gets all public information as input
and the move it can make will be just what the user can do minus the UI interaction. So if the user has to throw dices first by button click, the agent would just throw them as well. User selects 3 out of 6, agent does the same with setters
other things that might be added if the game was developed as person vs person, there might not be an input on how a computer could do a move (like everything is somehow done in the UI or grabbed from the board without a central game state)
I know that Dominion (the mobile or steam version) uses AI learning for the opponent
the game is really complex with all the card combinations (since there are like 10 expansions with maybe 250 unique card effects)
@Vaillancourt Since you're updating at frequent intervals anyway, it suffices to sample the curve at t + something and steer toward that. You'll pick a new target before you get there, so it doesn't matter if it's not the perfect distance away.
You can divide something by the magnitude of the derivative at t for something closer to uniform, or use a couple iterations of binary search to get it exact if your algorithm needs it.
