Working with HUGE database

Hello everyone,

I’m currently working on a program that let users write their own haikus and share them. To perfect it, I need to count the syllables and verify if the user’s entries match haiku’s structure. The program should be able to count syllables for each word and return a validation if the structure is respected.

I’ve already found a massive Excel array containing 140 000 words and their syllables count, but now I need to use it in GDevelop. So I’ve tried to convert my Excel array in json and paste the result inside the json of my project, in the “variables” section, following another topic.

The problem is that my json file is 12mb now, and of it won’t open in GDevelop.

So I thought about creating and loading into GDevelop an external dedicated json file stored on my PC. The idea is that for each word, GDevelop will pick datas in this json file and create a variable named as the word and containing the syllables count, so that I would only declare variables for the words effectively used, and not for the 140 000 in the database …

The thing is that I can’t manage to communicate with an external json file or whatever I could use in order to do this. I’ve also tried to follow another topic, but I needed to fulfil the path parameter in the Send a request to a web page action, and it seems that this parameter no longer exist since I can’t find it.

Here is a screenshot of a sample of my json file containing the variables. It’s the one I tried to include in the json of my GDevelop project :

Do you know how could I achieve this ? Is there a simpler solution which I didn’t think of ? Any help, advices or explanations would be appreciated :grin:

Thanks, and have a good day :slight_smile:

1 Like

Would my extension help you?

You appear to have a few repeated entries. Is there a reason for this?

Also, every section has a "type":"number" entry. If this is the same for all sections, you could just remove that and cut the file size down by about 30%.

You could break the files down into the first letter - so all names starting with a in the A file, all names starting with b in the B file and so on. Then read in the file when you need to check for the word.