Buttons and Text accessible by Screen Readers, Voiceover software, and tabbing

I would like a way to create buttons and text that can be accessed by screen readers when exporting to HTML5.

The games i create have to pass AA WCAG accessibility requirements. However, because GDevelop currently exports everything to the canvas, none of the sprites nor text objects get picked up by the browser. I was faking the tab functionality before, but I couldn’t fake the voiceover part. Then i realized that PixiJS does have some built in accessibility features.

I’ll explain below how I have made buttons and text accessible in GDevelop but it is super “hacky” and has a chance of breaking every time there’s a new version. The code I use should definitely not be implemented, but I really want the functionality to be. Doing this makes button-sprites become actual buttons in the browser with their own html-button tag and attributes.

So Here’s how I do buttons:

  1. I make sure every button has these object variables.

  2. I use an event like this to trigger when the enter/return button is pressed
    Screen Shot 2022-03-31 at 2.13.59 PM

  3. I then add this code to this to the _updatePIXISprite() method in the spriteruntimeobject-pixi-renderer.js file

var accessibleButton = this._object._variables._variables.items.accessibleButton._bool;

            if (accessibleButton) {
                this._sprite.interactive = true;
                this._sprite.buttonMode = true;
                this._sprite.accessible = true;

                this._sprite.accessibleTitle = this._object._variables._variables.items.buttonTitle._str;
                this._sprite.accessibleHint = this._object._variables._variables.items.buttonAriaLabel._str;
                this._sprite.tabIndex = this._object._variables._variables.items.tabIndex._value;

                this._sprite.click = function(e) {
                    var customCanvas = document.getElementsByTagName('canvas');
                    customCanvas[0].focus();
                    if (typeof customIndex !== 'undefined') {
                        customIndex = this.tabIndex; //update customIndex defined in global scope from customIndex.js
                    }
                }
                
            }




For Text this is what I do

  1. I make sure that text that needs to be accessible has the following object variables
    Screen Shot 2022-03-31 at 2.17.39 PM


  2. I add the following code to the updateAngle() method in the bbtextruntimeobject-pixi-renderer.js file
if (this._object._variables._variables.items.hasOwnProperty('accessibleText')) {
                var accessibleText = this._object._variables._variables.items.accessibleText._bool;

                if (accessibleText === true) {
                    this._pixiObject.interactive = true;
                    this._pixiObject.buttonMode = false;
                    this._pixiObject.accessible = true;
                    this._pixiObject.accessibleType = 'text';

                    this._pixiObject.accessibleHint = this._object._variables._variables.items.textAriaLabel._str;
                    this._pixiObject.accessibleTitle = this._object._variables._variables.items.textTitle._str;
                    this._pixiObject.tabIndex = this._object._variables._variables.items.tabIndex._value;
                }
            }

And to tie everything together:

  1. create a file called customIndex.js that has the following line of code:

var customIndex = -1;

  1. add the above mentioned javascript filed to the index.html
    <script src="customIndex.js" crossorigin="anonymous"></script>

  2. create a global variable in GDevelop called “customIndex” and have that constantly sync with the customIndex variable from our customIndex.js file.








    The final result is elements that sit on top of the canvas that can be seen by the browser and therefore can be seen by accessibility functions and programs as well.

Screen Shot 2022-03-31 at 2.33.51 PM

1 Like

Good job :clap:
But what kind of games do you build for visually impaired people? :slight_smile:

I make Security Awareness Training games. And the company I work for definitely wouldn’t mind working closely with the GDevelop developers on this feature request. We love GDevelop with all of its capabilities and simplicity. But accessibility is very important to us.

1 Like

Wonderful hack!

You use lot of private scope like: this._object
Did you know GDevelop have documentation for the API :slight_smile:

1 Like

You’re very welcome to contribute to the GitHub, you can clone/fork the source code and request a merge when your code is ready.
If you’re not sure how to implement the feature, open a thread there to discuss the proper implementation with the dev team. :slight_smile:

2 Likes

So there were a couple of reasons why I did it that way. But please let me know if you have suggestions/alernatives.

  1. biggest reason was so that our developers wouldn’t have to implement the logic on their own for every button in the game. By me setting a “flag” (a boolean object variable) I can then do all the work for them after they export the game.

  2. I do use this._object to get that flag i mentioned above, but what I am really manipulating is this._sprite. And I don’t think that is exposed using the method you linked. (But again correct if I’m wrong, because that would be a lot easier.)

  3. I did try to include the the method you link, but I couldn’t figure out how to pull in the RuntimeScene object into that _updatePIXISprite() function. And again, I’m doing my work there because I need access to this._sprite which I don’t think is exposed anywhere else.