[Solved] Issue searching for newline character with StrFind

Bluzima · August 28, 2022, 5:56pm

Hi,
I often use StrFind function to look for a string in a text, (this text I often load from a .txt file in folders).
It works fine, except sometimes, with the return (line break) character, that cannot be found (sometimes).

Is there anything specific to know about this character? Does gdevelop StrFind function works in a specific way for this character?

Thanks!

MrMen · August 29, 2022, 2:16am

Have you tried StrFind(VariableString(someString), NewLine())?

Bluzima · August 29, 2022, 1:18pm

Thanks, I’ve never tried that, didn’t know that “NewLine()”.
I’ll do, and will give more information if I understand why is my issue happening.

Bluzima · August 30, 2022, 1:55am

Hi again, I run some tests to understand what doesn’t work well. The problem doesn’t seem to be in the newline character, but in a star “*” followed by a newline … ! I don’t unerstand why, here are the description and results of my tests :

Tests:

In the 3 tests, I look for the same string that is a star followed by a new line
What change in each test is in what string we are looking for.

Test 1 : we’re looking inside a string that is a variable where a text from a .txt file has been previously loaded.
Test 2 : we don’t use a variable, we’re looking inside a string directly, this string is a copy of the text in my .txt file
Test 3 : we’re looking inside a variable that has been set to a copy of the text in my .txt file

Result of the tests :

I run 3 preview : one looking for the star alone ("*"), one looking for the newline alone, and then one looking for a star and a newline.

if I look for a star only “*”, or if I look for a new line only , it is found in the 3 tests.
if I look for star and new line together, it’s not found in the first test.

Note : I opened debugger to look at the string loaded in test 1. I converted this string to binary, to compare with my .txt file. But it seems to be the same.

So, the problem seems to be :
looking for a star followed by a newline doesn’t work when the main text has been loaded from a text file.

I just share the result of my test, and if I did no mistake, it might be consider as a bug… ?
It’s kind of a specific issue though, a few people only might encounter this problem.

Bluzima · August 30, 2022, 1:57am

I just run a new test, replacing the star by another character, “l”. The issue is the same.
So it is not about the star character.

As far as I know, the problem can be generalised as:

" Looking for a character followed by a new line doesn’t work when the main text we’re looking in has been loaded from a .txt file. "

So the issue is larger and can be encounter by more people, since you might want to look for things in a .txt file in paragraphs separated by new lines …

Bluzima · August 30, 2022, 2:02am

I just replaced in test 1 "*
" by : “*”+NewLine().
The result is the same, it cannot be found.

MrMen · August 30, 2022, 2:45am

I don’t know what you’re doing, but here’s my test, looking for “*”+NewLine(), and it works. My variable setup :

The value of str_pos in the debugger inspector :

Have you checked the characters in your file (a hex editor would be ideal)? It could be the line is terminated with the old CR + NL (Carriage Return and New Line), so you may need to search for CR (ASCII 13) instead.

Alternatively, regular expressions (regex) may be of some use.

Bluzima · August 30, 2022, 3:10am

Thanks for your reply

What you did is my “test 3”. It works fine on my side too. What doesn’t work is what I did in test 1, which is, load a text from a file into a scene variable.

I can try that yes, but I’ve never used a hex editor before. What I did though is to compare the binary translation of the string I look for, and the actual string I’m looking in, as shown by the debugger, and the binary octets are the same.
So I do look for a string (character + NewLine) that should be here since the debugger shows it in the variable, but that is not found for a mysterious reason…

Thanks for the regex advice! I’ll give a try.
Nevertheless my tests make me think the problem appears only when loading from a txt file… So the trick might be in the loading event and not in the StrFind function…

MrMen · August 30, 2022, 3:37am

This is the variable holding the text that was read in? Then maybe someone from the GDdvelop developers might be able to shine some light on t…

One other option may be to use a javascript event to process the input string. Not sure how or what, but that may be able to handle the input a bit better.

Bluzima · August 30, 2022, 4:24am

Yes it is. I just checked again.
The picture here shows the comparaison I made :

on left side, the condition, lookin for string “* newline”
on right side the actual text variable where I’m looking in
The binary translation of the text looked for and the binary translation of the variable where the search is run, shows that the strings are the same. However the StrFind doesn’t find the string.

( I don’t know if to translate in binary is a good practice, but I did that copying the strings in an online translator, in order to compare what the characters actually are)

So I guess yes… I checked a few times and it seems that there’s a strange thing here.

Yes

Silver-Streak · August 30, 2022, 4:29am

Are you trying to search for wildcards or the actual asterisk character? Because wildcards aren’t supported in any way within expressions or variables. They are just treated as asterisks.

(Edit: Wanting to confirm this first before I do testing tomorrow)

Bluzima · August 30, 2022, 4:35am

No I’m not looking for wildcards. The asterisk “*” was an example. But I tried with the letter “l” and the result is the same.
When I load into a variable a txt file that contains only a character followed by a new line, the variable can be seen in the debugger, well loaded.
But looking for this variable string value, in this variable string value return - 1 (strange!?!)

Silver-Streak · August 30, 2022, 4:39am

Okay, I’ll test tomorrow. 99% sure you cannot use return/new lines in expressions without the format MrMen showed, and I’m using some loaded texts in my test projects that search for special characters including newlines.

I’ll see if I can reproduce it and report back, if so we’ll get an issue on the GitHub

Bluzima · August 30, 2022, 4:40am

I tried a few times, trying to isolate the issue with tests… I can do some more, I don’t want to make you loose time with this, but I don’t know what to test more…

Bluzima · August 30, 2022, 4:49am

I did one more test, this time using NewLine() format instad of the newline character. It does the same.

My .txt file is nothing more that the letter m and a new line
I load it into a variable “text”

if I look in this variable text for the letter m alone, it works, my condition compares 2 numbers : StrFind(VariableString(text), “m”) =/ - 1 is true
if I look for NewLine() alone, the condition StrFind(VariableString(text), NewLine()) =/ - 1 is true too
the problem occures when I look for the letter m followed by a new line. The condition StrFind(VariableString(text), “m”+NewLine()) =/ - 1 is false …

( I wrote " =/ " for " not equal to " )

Bluzima · August 30, 2022, 5:15am

One more test I just did :

The problem is specific to any string where a character is followed by a newline.

NewLine() +“m” works fine
“m” + NewLine() doesn’t
“m” + NewLine() + “something” doesn’t either

Silver-Streak · August 30, 2022, 9:14am

Woke up a way too early and did some testing. I can reproduce this a few different ways, so I’ve opened up a Github issue with a test project:

github.com/4ian/GDevelop

New Lines/NewLine() Expression are treated differently if stored in variables loaded from external text files.

opened 09:13AM - 30 Aug 22 UTC

Silver-Streak

🐛 bug

## Describe the bug Bluzima on the forums is trying to load data from a text fi…le into their game, then test if there is a NewLine in it to do an action. They are using the expression StrFind(VariableString(Loadedtextvarhere),"*" + NewLine()), which should test fine if the text file has any instance of "whateverprecedingcharactersyouwanthere * (newlinehere) whateverelseyouwanthere" However, if loaded from a text file and stored into memory, while it will display correctly if you load that string variable into a text object, it *will not* evaluate correctly. The same is true if you try to use the TextObject.String() expression to test against, instead of a variable. What *does* work is if you manually populate a string variable with the text I pasted above. It will evaluate fine. ## To Reproduce Steps to reproduce the behavior: 1. Create a text file with whatever characters you want, followed by a new line (return/enter key). 2. Save the text file somewhere you can easily build the Expression-based path to in GDevelop. 3. In a new GDevelop project, load that text file into a variable using the "Load text from _ into scene variable _" 4. Set up a new event that uses the "Compare two values" condition, and tests StrFind(VariableString(YourVariable),"yourcharacter" + NewLine()) is greater than -1. 5. Set up an action to do anything. The action will not occur. There are a few permutations to this, but things Bluzima and I have now tested: 1. String Find against a string variable loaded from a text file (above scenario) - Doesn't work 2. Modify the string of a text object with the variable from 1 above, then String Find against the TextObject.String() expression. - Doesn't work. 3. Create a string variable (via the variable editor or events) with the text copy+pasted from the text file, then String Find against that scene variable - *DOES WORK* 4. Modify the string of a text object with the variable from 3 above, then String Find against the TextObject.String() expression. - *DOES WORK*. Everything in red in the screenshot is done with the variable that was created by reading from the text file. Everything in blue is done with the variable that was manually created. ![image](https://user-images.githubusercontent.com/24237149/187397882-8d509443-2d03-4b54-a185-adf3947ae194.png) Here's the example project. You will need to extract it to your desktop for the file path to work. [NewLineTests.zip](https://github.com/4ian/GDevelop/files/9451486/NewLineTests.zip) ## Other details * Include any OS/browser version/smartphone that you're using - Win 10 64 * Which version of GDevelop are you using? The desktop app or the web-app? - 5.0.140 * Add any other context about the problem here. - Forum thread here: https://forum.gdevelop.io/t/issue-searching-for-newline-character-with-strfind/41814/15

As with any issue, there isn’t a timeline on when/if it’ll be fixed, but hopefully the sample project can assist narrow it down for the devs or contributors that look at it.

Bluzima · August 31, 2022, 7:19am

Just had time to look at it. I read your github post, that’s exactly what the issue is about.
Thanks a lot!

alexandresi · October 3, 2022, 3:21pm

Hi,
I found an explanation for this issue (See New Lines/NewLine() Expression are treated differently if stored in variables loaded from external text files. · Issue #4246 · 4ian/GDevelop · GitHub).

I think the best is to add a new expression that would be NewLineForWindows or something similar.
Or I could create an expression that cleans a string from the \r characters.

Do you have an opinion on this?

Silver-Streak · October 3, 2022, 3:27pm

I think something that cleans \r (or replaces it with newlines) may be the most flexible option? Especially since someone could use it on a string that doesn’t contain \r (if they’re unsure) and it wouldn’t cause any issues, correct?