[Added] Asynchronous instructions in GDevelop with wait action and delayed custom actions

Asynchronousity is not handled the right way in GDevelop. Asynchronous libraries are used by making them store return values in some array to get it synchronously later. The main disadvantage is that you either have to skip some data or delay it’s processing to the next frame when you get data twice or more from the same async event, making it inacurate by either missing a value or using it too late.

The best way to handle callbacks is to let user s decide what code will run as callback of the async function, by either passing an external event as callback or having a system like external events/functions (but just for callbacks) to instructions as parameters, bascically passing a function as parameter of functions.

The way is see it is something like functions, but more standalone (no need for an event based extension), and callbacks parameters would be defined ≈ like this

const callbackMultiplayerConnected = extension
  .addCallback("Connected")
  .addArgument("string")
  .addArgument ("scenevar");

extension
  .addAction(...)
  .addParameter("string", false, "", "") // PARAM0
  .addCallback(callbackMultiplayerConnected); // PARAM1

// Alternative

extension
  .addAction(...)
  .addParameter("string", false, "", "")
  .addParameter("callback", false, "", "")
  .setDefaultValue("Connected"); // Callback name

To pass a callback to an instruction the parameter types have to match (if the types doesn’t match the callback is greyed out and put at the end of the list, hovering over it with the mouse tells where the incompatibility is)

I get the idea, but how would you see this concept materialise in the events sheet?

Ok I thought about this thoroughly again and made this “proposal document”:


0. Some base implementation details:

First, note that the scope of this feature request has moved from passing “functions” to generally executing asynchronous code. Passing functions would be nice too, but asynchronous is more important right now because of how much it is used in the JS ecosystem and permits in general a more fluid experience while using HTML5 based games, JS being generally single threaded.


The implementation details aren’t fixed yet, but a few things are pretty sure:

  1. The async action shouldn’t run if the scene is not on the scene stack anymore as that means the scene is deleted.
  2. The async action has to be able to return some data.

For point 1, I’ll be adding a check in the code examples if (!runtimeScene.getGame().getScneStack().hasScene(runtimeScene)) return;. This method doesn’t exists, but you can interpret it as:

for (let scene of sceneStack.stack) {
  if (scene === runtimeScene) return true; // Compare references
}
return false;

For 2, I will use a “result” argument for the callback. This post isn’t about how to get the result of the async action but about how to make async actions in GDevelop. The result will be discussed later (in another post or after deciding on the method here).


Another note, most of the proposed methods here would work especially well if a scene can have multiple event sheets. That would help split the code in more readable modules easily and have some callbacks as separate event sheets but that are still “part of the scene”. Of course external events can be used but personally I like the idea of organizing what belongs to the scene in the scene and keep external event sheets for “global” code.


1. “Event Emitter” like:

Description:

The runtimeScene would possess a HashMap of event handlers. The user selects event sheets via an interface similar to the link event, and they will get exported as functions and stored into the scene. When an asynchronous action finishes, the developer calls the function that corresponds to his callback.

Engine developer experience:

The experience should be pretty OK but would require researching for quite a bit. The developer would define a callback with some API similar to this:

extension.addCallback(
  "OnFirebaseAuth" // Callback name
)

Then, he would call

runtimeScene.dispatchEvent("OnFirebaseAuth", results)

To call the callback after the asynchronous action bound to it is finished.

Mockup:

Pros and cons:

Pros Cons
No callback hell Isn’t intuitive
Hard to find
Not very clean

My comment:

This was my first idea and isn’t a very good one, but I still include it to show really all the possibilities. It is OK in a way I guess but the 2 other possibilities are better I think


2. “Callback” like:

Description:

We would make it possible to pass event sheets (the same inputs as the link event) as parameters of functions. It would be generated as an anonymous lambda function as a parameter of the Instruction.

Engine developer experience:

The generated code would look like this:

action(
  function(){
    if (!runtimeScene.getGame().getScneStack().hasScene(runtimeScene)) return;
    // Insert all the events passed
  },
  ...parameters
);

This should be pretty developer friendly, as it’s the most intuitive method to use. Just pass a callback, define it as argument, and call it once the async operation ended.

Mockup:

With links:


With direct event:

Pros and cons:

Pros Cons
Very Intuitive Doesn’t look good
Hell to chain (callback hell but worse :frowning:)

My comment:

This is the most easy to understand/intuitive solution, but it isn’t suitable for more complicated operations/in the long term.


3. “Promise” like:

Description:

We would have a new instruction type “AsyncAction”, that you would call with a special Async event. The async action is like a normal action but has to return a promise.

Engine developer experience:

The exported code would look like this:

asyncAction(...paramters)
  .then(results => {
    if (!runtimeScene.getGame().getScneStack().hasScene(runtimeScene)) return;
    // Insert all the normal eventCode
  })

Apart from returning a promise and defining as async the method, the developer doesn’t has to do much and it is pretty easy to deal with.

Mockup:

Pros and cons:

Pros Cons
Clean in the event sheet Hard to find
Easy for the developer to implement Hard to understand
Chaining is easy and readable Uses promises which might not port well to other languages/platforms

My comment:

This method is pretty effective: not really hard to use, even though it’s not the easiest either, and fits well in the event sheet. My only concern is portability across other platforms/languages. This is the method I would go with anyways.


While I don’t know of any instances where I’d personally use async events for my game, your proposal makes sense to me on why there could be value there.

As far as the options, #3 seems the most like it matches with the existing design language of the event sheet, while #2 seems like it may be easier to manage?

As far as your comment about multiple event sheets, would something similar not be possible just by using event groups? (With names like “Event Sheet 1” “Event Sheet 2” etc)

Yes, kind of. I think the Link event even supports including just a group instead of a full event sheet.

Overall there are interesting ideas :slight_smile: I’ll comment on a few things then will give you my idea.

Passing functions would be nice too, but asynchronous is more important right now

Right, in the model of GDevelop events, asynchronous is more important, I’ll explain why.

You might also want to retain the picked objects. Much like a callback/promise in JS can read the values outside of it (it’s what is called a “closure” in some programming languages, as it can “capture” references to variables outside the function), we would need to remember which objects were selected, to be able to naturally continue operations on them.

because of how much it is used in the JS ecosystem and permits in general a more fluid experience while using HTML5 based games, JS being generally single threaded

JS is not really the reason, nor the “single threaded” limitation. Asynchronous is interesting because it allow to set up a set of tasks that will be run and continued from where they stopped.

As you explained we’ll avoid “Event Emitter” like. It looks a bit cumbersome to use as you noted.

Could be interesting, but not sure it “scale” when you’re starting to make complex events.

This method is pretty effective: not really hard to use, even though it’s not the easiest either, and fits well in the event sheet. My only concern is portability across other platforms/languages. This is the method I would go with anyways.

Promise is the most interesting of these 3 solutions indeed, but we would not necessarily use the promise abstraction from the JS language, but we could emulate it.


Now, I have yet another idea :slight_smile: Event emitter/Callbacks/Promise are all missing an interesting concept, which is now in JS in the form of async/await: the ability to write code sequentially, but have it to run asynchronously. Under the hood, this is actually based on promise, and you could argue it’s actually “just” syntaxic sugar on top of promise. But it’s very nice sugar :wink:

If we go back at GDevelop, what are the cases where we might want to have something that is “asynchronous”? It’s when basically we have to wait. It can be because we’re waiting for a response of a web server, or because we wait to add a pause in a process, or because we’re waiting on something else.

The notion of wait also implies that when the wait is over, we want to continue where we left. This is where it’s important to “remember” the picked objects: you want to continue working like if there was no wait (well, of course if some objects were deleted, you can’t operate on them. But apart from that you can continue with the objects).

Basically, I’d like to find a way to write something like this in the events sheet:

if (some conditions) {
  await launchAWebRequest();
  doSomethingWithObjects();
  await delayOf5Seconds();
  doSomethingElseWithObjects();
}

The creator could choose to use external events if you insist, but overall I think that the notion of “wait” should be almost transparent. Maybe we could have asynchronous actions that would stop the event processing and continue when they are done:

Conditions: 
  some conditions
Actions: Launch a web request with X, Y, Z parameters
  Set X position of Object 1 to 100
  Wait for 5 seconds
  Set X position of Object 1 to 200

In yet other words, my proposal is that any action can be asynchronous.
This will need changes in the code generation to:

  • flag an action as asynchronous (easy)
  • persist the context, i.e: the selected objects and… that’s all? This should be persisted in some place, surely a hashmap/object that would associate to a “context id” the list of selected objects (and any other thing we would need).
  • then stop there.
  • then when the action calls a callback (or returns a promise?), the code generation would call a function to “restore the context” (passing the context id as parameter). It would then call the function corresponding to the remaining actions, after setting back the object lists.

This would also need to change the generation of actions, so that when there is an asynchronous action that is generated, then all the next actions are generated in a separate function. Note that for sub events… it’s already generated in a separate function so it works already :slight_smile:

Much like the way JavaScript engines are working, when an action callback is called, it would be added to a list of “contexts to resume”. All of the contexts to resume would be executed, then the scene events as usual. Note that it would work with multiple asynchronous actions (after resuming after the completion of the first one, there could be another one, that will do the same “dance”: save things in a context, call a callback or return a promise when done which will resume the context).

There would be remaining questions:

  • can conditions be asynchronous? (it would be yet work code generation rework, not sure if a good idea… though could be interesting in the future to compute things like pathfinding in a web worker/other thread??)
  • is there a risk of confusing the user? Probably asynchronous actions should be flagged. An action like “Wait 5 seconds” is very intuitive, but maybe things like web requests should be written “Launch a web request AND WAIT BEFORE CONTINUING” (not all caps, but you get the idea ;)).

I think overall it could make a very clean result in the events sheet, and avoid most needs of these “separate events that you could pass as parameters” like in the evens emitter or the callbacks. If you really want them, just make an external layout… or an extension function :slight_smile: That’s what extensions are for: they are really functions, except that what we call functions in programming languages, we call them “actions” (or conditions) in GDevelop.

That’s it! What do you think?

:thinking: the async await solution is so evident I am ashamed to not have thought of it as well. It indeed sounds like one of the cleanest solutions in the event sheet. Though I personally like a little bit the promise approach more because it makes it very clear when and how asynchronous code is being executed. For example if someone wants to execute some code after the asynchronous action is executed and some directly, they would add a subevent with as first action the asynchronous one and directly after another subevent with all the code to be executed immediately. This may sound ok, but for newcomers this would be very confusing (why is there a separate subevent for those actions? Why does it break when I move it that way?), when the promise event would directly says that something special is happening here, and they would only have to click on the ? Icon or search for asynchronous in the wiki to see what is going on.

Personally, I think the way to go would be:
Adding the promise event AND as syntactic sugar on top of it an await action that takes an asynchronous action as subinstruction. This technique is sort of proven as it is the one used in JavaScript. This makes it both obvious that something is happening (imagine if asynchronous functions were automatically awaited in JavaScript, this would be making much code much more confusing) and also pretty clean to read in most cases.

I thought about this too, but decided to not include it in the post because I think it would be problematic, as the picked objects might have been deleted, as we probably don’t want the actions to affect a deleted object if another non deleted one is also in the picked objects list. I am not 100% sure how object picking works, but I am pretty sure from when I read the generated code to restore some lost events that it doesn’t recheck if the object still exists before making use of it, as the objects are only actually deleted between event sheets executions. So we would need to recheck every picked objects to see if it still is in the scene which would be a very expensive operation especially on a big games with many objects.

tldr; I am unsure if it is a good idea so I didn’t put it on the list of things that are sure about asynchronous code

Personally, i think that their scope is much more limited but they would allow some much asked features (mainly the wait condition from construct).

I’d suggest to progressively roll out all of this:
PR 1: Refactor code generation to support asynchronous actions and the promise event
PR 2: Add to all asynchronous actions in GDevelop a real async version and hide the old
PR 3: Add async await syntactic sugar
PR 4: Add async conditions

That way the huge feature would be able to be tested over time as we finish the “optional” parts that are not necessary to asynchronous code in general.

Then, I also understand the point of trying to make asynchronous as “invisible” to the user as possible, but I really think that it’ll confuse everyone more than making it easier. Tell me what you think!

I agree that it should be made obvious when reading the events sheet that something will be “awaited”, much like the await keyword in JS.

But I’m afraid that a “promise event” or an action taking a subinstructions are too “developer oriented” and miss a bit the simplicity we want for users. Also in JavaScript, while we can not put await, it’s almost always an error not to do so.

I’m thinking of something like this quick mockup:

It works like this:

  • actions declaring themselves as async will be automatically “awaited” by default, with a “chip”, like on the mockup, showing it clearly in the events sheet.
  • if you don’t wait for the result, you can in the action editor disable a checkbox “wait for the result” (which is checked by default) (so we still give the choice… and keep compatibility with existing projects!).

Note that an asynchronous action like “Wait for X seconds” can’t have its “wait for the result” unchecked :wink: Note also that the label “and wait for the result” could maybe be customized.

PR 1: Refactor code generation to support asynchronous actions and the promise event
PR 2: Add to all asynchronous actions in GDevelop a real async version and hide the old
PR 3: Add async await syntactic sugar
PR 4: Add async conditions

I think we should have this “syntactic sugar” to be the default, so probably have the first PR about refactoring code generation tu support asynchronous and add the same time support for displaying it in the events sheet (or at least in the next PR).
Then as you said the PR to make GDevelop actions async. Note that existing actions in extensions can be marked async, but by default for existing projects we don’t check the checkbox “await for the result” (which is surely a new boolean in a Instruction: await to be set to true/false).

I think the “promise” event is then not really necessary? Just create an event without conditions and an action to be awaited?

Conditions are clearly not a priority and can be a last PR.

That way the huge feature would be able to be tested over time as we finish the “optional” parts that are not necessary to asynchronous code in general.

That’s the idea, going step by step and keeping the not so necessary feature at the end :slight_smile:

1 Like

Adding this because I forgot:

In this approach, it’s essential that we can keep track of the selected objects - because the whole abstraction of the events sheet is based on that :slight_smile:
When an action is await and its context (i.e: its list of objects) is “saved”, there is indeed the need to remove any destroyed objects from it when an object is deleted. It should be doable because when an object is removed, markObjectForDeletion is called. At this moment, or maybe when _cacheOrClearRemovedInstances is called, we can iterate on the “living contexts” (i.e: on the list of actions being awaited) and remove the instances being removed from it.

That should be it? Objects created while an action is await are not added to the contexts - but if an action later use objects that were not selected before, it will automatically ask the scene for the objects (using getObjects), which will properly return the created objects.
So we only need to care for the deletion of objects, and for the saving and restoration of the object lists. The rest should already work - because the logic of object selection does not depend on if an action is asynchronous or not.

Hide Loading is trigger only if the request action awaiting respond ?
The action trigger after the repond should be more visual, like in a group or a special type of subevent, atleast visually.

Like, you get the idea.

Hide Loading is trigger only if the request action awaiting respond ?

Correct. Precisely, it’s called WHEN the request action is finished. In other words, it’s called after the request is sent and the response was received (whether it’s a success or not).

The action trigger after the repond should be more visual, like in a group or a special type of subevent, atleast visually.

It’s a possible approach, though I wonder if we could avoid having a “group” with indentation.

What do you think of something like this:

It should make it clear that after the action, there is a “pause” while something is being completed.

We can make the separation a bit more obvious with a dotted line:

This could be useful for making timelines, like:

Set animation of "Player" to "Run"
Wait for 5 seconds -------------------------------------------
Set animation of "Player" to "Shock"
Set text of Dialogue to "Wow, there was a wait in the actions"
Wait for 2 seconds -------------------------------------------
Set animation of "Player" to "Happy"
Set text of Dialogue to "It's pretty useful!"

It also goes with the idea in languages like JS that have async/await keywords of being able to write things like if it was synchronous, without extra indentation.

@arthuro555 Note that for “promise like” stuff, we could replicate a Promise.all action by making an action: “Wait for these actions to finish”:

Wait for these actions to finish:
  Wait 5 seconds
  Send a GET request to "http://example.com" with body "..."

This is a special action (like special conditions OR/AND/NOT) that would be waiting for the result of both actions (here it’s useful to wait for at least 5 seconds, even if the request is very fast).
Note also that this is getting into advanced stuff. For 99% of users, I think having just actions that are asynchronous and where the events are “paused” before continuing should be enough… and hopefully not too complex.

2 Likes

Yes I think this solution is good as well. For the implementation, my first idea was something like the actions code generator checks if the instruction is toggled to async and if it is, increment a promiseDepth variable in the context and instead of a semicolon put

.then(() => { // Begin of async code
if(runtimeScene !== runtimeScene.getGame().getSceneStack().getCurrentScene()) return;

And in the events code generator add a step

while(actionsContext.IsInPromise()) { // equivalent to context.promiseDepth < 0
    outputCode += "}); // End of async code";
    actionsContext.LeavePromise(); // == promiseDepth--;
};

But there are two problems.

  1. The action code generation only has access to the metadata not the instruction itself. (Is not a huge problem can probably be changed)
  2. The { } around the actions would close the function early and cause a syntax error.

The other method I see would be making the function of the event async and prefixing the async actions with await. I don’t know the async/await syntax well, but I am pretty sure that it would execute the whole event as a Microtask, where the behavior we want (I think) is execute all the instructions normally until the async instruction is reached, and only then schedule the rest as a Microtask to execute on the end of the async instruction.

So I see two options here:

  1. Remove all seemingly redundant { }
  2. Make the event functions an async function

I have therefore three questions:

  1. What are all the { } for?
  2. Am I wrong about the way async/await work?
  3. If there are synchronous actions before the asynchronous one, should they still be executed “asynchronously”?

The concern with “Chips” is that visually it has no impact on the resulting actions.
When I see this I think only that the answer to the request will be saved in a variable when it is possible.
I don’t see any link between waiting for the answer and the actions it should trigger.

The action group here would be perfect to understand this waiting system.

Users won’t understand anything if normal actions are not triggered because an await action is pending, all actions have the same style. (chips isn’t enough, there is no visual link between the await action and sub-actions)

“When the asynchronous action is complete, it return a callback and triggers the rest of actions.”
From what I understand it can be translated differently:
The asynchronous action is a kind of condition, because it returns a result and trigger the next part.
The sequence of actions are like normal actions.
So I make the parallel with a simple condition / action, we currently separate them, it would be unwise not to do the same.

Waiting a result is the same principe of trigger something with an action, the rest of the code/action is in pending in a separate function/group.

If you didn’t like the idea, i would see a more verbose chips…
“Wait for result (sub-actions are in pending)”
“Continue only if the result come”
“Pause the actions and wait for result”
Something like that.

In large scope, we should ask a pro in UX/UI for the whole game engine, there is a lot to do for make thing simpler, fix inconsistencies and more get visual IDE. I keep this idea for later!

Look you have done a indentation, it’s intinctive :stuck_out_tongue:

Even if this one is more logical to me:

Wait for these actions to finish:
  Wait 5 seconds
    Send a GET request to "http://example.com" with body "..."

So we have two stuff going on:

  • Choosing what it would look like
  • How to implement this

How it would look like

Look you have done a indentation, it’s intinctive :stuck_out_tongue:

No, I think you misunderstood how my example work.

I only have indention here because I have a special actions waiting on two others at the same time. This is a special case (I should not have mentioned it ^^) and will probably be almost never used.

The action itself “Wait for these actions to finish” would not be indented compared to others.

But this is not what I wrote :wink: Here in what you write, it would:

  1. Wait 5 seconds
  2. THEN it would send a GET Request.

While my example was:

  1. You start to wait 5 seconds AND you send a GET request (hence the indentation)
  2. You continue when BOTH are finished.

See the difference? It’s like in JavaScript:

// ...
await delay(5); // Wait 5 seconds
await axios.get(...); // Launch a request
// next stuff (no indentation)

vs:

// ...
await Promise.all([
  delay(5), 
  axios.get(...)
]); // Launch a request and wait 5 seconds, then continue when both things are done
// next stuff (no indentation)

I do agree that waiting for an action to finish is something that needs emphasise. But indentation proved to be something that in JavaScript and other languages was found to be cumbersome. And so await keyword was introduced.

Let me explain with another example. I’ll take the “wait X seconds” as the most simple “asynchronous action”. It would be useful to put some pause between two actions, great for timing stuff like animations or cut scenes.
If I write down as I speak, I would come up with something like:

  • Set animation of player to “something”
  • Wait 2 seconds
  • Show some text
  • Make a cool effect on screen
  • Wait 2 seconds
  • Play a sound
  • Play an animation
  • Wait 2 seconds
  • Go to the next scene.

See how my list of things (“actions”) has no indentation? :smiley:

If you didn’t like the idea, i would see a more verbose chips…
“Wait for result (sub-actions are in pending)”
“Continue only if the result come”
“Pause the actions and wait for result”
Something like that.

Yep I agree with that. Something like:

Maybe the “chip” is not a great idea, we can also alternate some background color to distinguish the actions that are “after” the wait:

Anyway, maybe we’re bikeshedding, but I think that it’s good if we can keep the indentation for events/conditions.
Asynchronous actions introduce a time delay, but not a flow change.

On the implementation

  1. What are all the { } for?

They are just there to delimitate every event, condition, action generated code. They should not be doing any harm, as any that is opened is then closed and actions are executed sequentially, not in the scope of the previous.

  1. Am I wrong about the way async/await work?

The main issue is that we can’t just use async/await. It won’t work like this, notably because we have scenes in GDevelop. Anything that is waiting must be scoped to a scene. For example, if I do:

Set the animation of player to "Run"
Wait for 2 seconds
Set the animation of player to "Jump"
Wait for 2 seconds
Change scene to "Main menu"

and in the meantime I go to a scene (i.e: in another event, I used the action to change the scene to go to “Final Level”, because the player won).

What should happen after 2 seconds? Nothing! :slight_smile: The “player” should not have its animation changed to jump… because I’m not even in the scene anymore :smiley:
What should happen after yet another 2 seconds? Nothing! I don’t want to go back to the main menu, because I’m not even in the scene where I ran these actions.

Even more interesting: If I pause a scene, play another scene and then comes back to the first scene, the actions “Wait X seconds” must have “stopped”. This means these actions timer must be updated using the game engine (not using a setTimeout).

In other words => because we’re a game engine, our “tasks in suspend” must be periodically checked to be launched at the beginning of each frame.

We can still code our extensions using async/await, in the sense that we can make an action that is a Javascript async function (or a function that returns a Promise - this is equivalent).
But the generated code should do something like this:

function myActions() {
  myUsualAction1();
  myUsualAction2();
  runtimeScene.addAsynchronousTask(myAsyncAction(...), mapOfObjectsList, theRestOfMyActions);
}

function theRestOfMyActions(mapOfObjectsList) {
  // Restore object lists from mapOfObjectsList
  myUsualAction4();
  myUsualAction5();
  // ...
}

At the beginning of each frame, the runtimeScene would iterate on the “tasks”. If one has a promise that is completed (or errored), it will call the function to continue.
When all tasks are executed, events are triggered as usual.

  1. If there are synchronous actions before the asynchronous one, should they still be executed “asynchronously”?

No, they should be executed as usual.
If I change the player animation, it’s not asynchronous, so the action is executed, and that’s it.

Only the asynchronous actions are doing something special:

  • they launch their code
  • and the promise that they return is saved in a “asynchronous task in the scene”, along with the context. The context is : 1) the selected objects and 2) the function to call when it’s finished.
  • the “function to call when it’s finished” is the generated code of other actions. Note that if one of these actions is asynchronous… well same thing! :smiley: (promise must be saved with the context, then another function is generated for actions after it).

Note that I’m opened to other solutions :wink: Like an “arrow”, a dotted line, a big bold text, or even vertical dots like this: ⁞ to make a visual separation of the actions before the asynchronous wait, and the next actions.
Or even have actions in two “lists”, that are connected by a dotted line. I agree that it must be clear that says clearly “there will be a pause here, and it will continue some time later”.

Hello ! Sorry for the necro… Has this discussion gone into development somehow ? I might make use of it…

It is live in GDevelop currently :slight_smile: That is what the “wait” action among other things is based on.

1 Like

Ah ? :thinking: I’ll have to try a few things then…
Does the “Wait” turn the event/subevents into fully asynchronous ones (my appologies if my vocabulary is too vague…). Which means that thoses events execution won’t delay the others. Or when they are executed, they will stop all event of the current frame until they are finished ?

EDIT : Ok. I have my answer (I have made some tests). Only the “Wait” action is asynchronous. It does not make the event et subevents asynchronous, to the rest. So when that event starts (after the wait delay), it is executed, stopping everything else until it is finished.